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PREFACE 

This is one of 10 units in a program of Training for Leadership in 
Local Educational rmprovement Programs . Development of the program was ' 
begun at the Learning Research and Development Center at the University of 
Pittsburgh and has been carried forward at Research for Better Schools in 
Philadelphia. 

If you nave in hand. the Instructor's Guide to the program, or Unit 1 
entitled Training Program Introduction, and General Study Plan Guide » you 
v/ill have sufficient introduction to the nature and purposes of the 
training program. If you do not have access to one or both of these items, 
the foUowing'paragraphs will introduce you to this unit of the^ program. 

This unit was designed for use by anyone holding a position calling 
for leadership in planning and conducting local educational change programs. 
This means school district leaders - central office administrators, building 
principals, curriculum specialists, or teachers involved in changt project 
teams. Also it means graduate students in curriculum, administration, or 
supervision. In addition, curriculum specialists or field personnel of 
state education departments or other educational agencies may find the unit 
of vahje in their work with school districts - as in the conduct of workshops 
involving local school personnel. 

The unit can be studied on a wholly self-instructional basis, or with 
an instructor's direction. It requires one or two days of study time. 

The unit*s topic - improving the implementation of a local educational 
program - is clearly an important one. A program cannot deliver its potential 
benefits unless it is effectively implemented, and studies have shown that 
ineffective implementation of local programs is a contmon occurrence. 
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UNIT 9. ANALYZING AND IMPROVING THE IMPLEMENTATION 
OF A LOCAL EDUCATIONAL^ PROGRAM 

Introduction 

No educational program, however well designed, will be successful unless 
It Is effectively implemented.- This is just as true off'''*trad1tionaV* 
programs as of innovative or change programs that are newly Introduced In a 
school system. Very often, what a school system should undertake Is improving 
the Implementation of traditional programs rather than Introducing new 
programs. The saine Is true of a change program that has been underway for a 
reasonable period; often the need Is to make Its Implementation more effective. 

Studies of the degree of Implementation of change programs have revealed 
that a high proportion of such programs never is fully Implemented. This has 
been found, for example, v/lth change programs Introducing nongr^aded 
organizational plans, cooperative teaching, departmentalization in the 
elementary school, and Innovative curricula In elementary and secondary^ 
schools. 

Often a program Is Implemented well In Its^ formal aspects (materials, 
equipment, staff assignments j etc.j but not in the conduct of instruction or 
oth'&r procedtiPSs It calls for. Thus, abtllty grouping usually Is well 
Implemented In the assignment of students to groups since this requires only 
administrative arrangements. However studies have found that many teachers 
do not modify their Instructional approaches to suit differences between high 
and low groups. Innovative curricula adopted by a school system very often 
are poorly Implemented because teachers teach them using traditional 
Instructional approaches instead of the new approaches required. 

Improving the ImpleEnentation of programs that already are in operation is 
a vital function of specialists In educational change. Until it achieves 
effective implementatton of a program, a school district cannot reap the 
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benefits, that the program was designed to yield. Also, no fair judgment, can 
be made about the potential benefits of a progc^m whose key features have 
npt been placed effectively in operation. 

The term "local educational program" as used in this unit requires 
definition. The temi is used here to refer to a school's "instructional 
program or any major segment of ^t. Thus, the self-contained classroom and 
team teaching represent general organizational arrangements for instruction, 
while the use of the Sullivan Basic Reading Laboratory (BRL) materials 
illustrates a part of the instructional program in one curriculum area. The 
term also refers to any planned provis^ions for interpersonal or intergroup 
relations^at school or for school-coinmunity relations. In addition, it refers 
to any aspect of staffing^ administration or supervision, and staff training* 

Unit Objectives 

In this unit you will study how to identify key features of a local 
etiucational program and how to assess present levels of implementation of 
those features* You will learn how to probe for causes of faulty 
implementation and practice the leadership function of designing plans to 
improve the program's implementation* 

Upon completing this unit, you should be able to: 

1. Identify and list the essential features of h local educational program 
from a written or oral description of it. 

2. Design a plan for assessing the implementation of the features of an 
educational program. 

3* Design a plan for conducting a causal analysis of shortcomings in 
the implementation of an educational program. 

4. Design a plan for identifying and evaluating resources likely to 
overcome the shortcomings identified. 

5* Design a plan for improving the implementation of an educational 
program* 
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Who Can Benefit from Studying This Unit ? 



This unit is meant, especially Tor school system leaders who have 



responsibilities in planning and implementing local educational programs,- 
particularly associate or assistant superintendents, curriculum coordinators, 
building principals^; and members of project teams. The 'unit should be 
valuable both for incumbents in these roles and^for those prepaWng for such 
roles* In addition, the unit should be useful to consultants from state 
education departments, educational consulting firms, and universities who 
frequently are called upon by school systems to help them improve their 
capabilities and performance. 
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Unit Study PUn 

In approaching this unit your fic'st task is to decide how intensively 
you v/ant or need to study each objective. Then you will need to plan how to 
go about it. Here is a guide for doing so, eiHIier with help from your 
instructor (if you have one) or on your own. . The procedure you are to follow 
is in four steps; assessing vour needs to study the unit, deciding on your 
way of studying it, assessing your mastery of the unit objectives after * 
study, and filling out your evaluation of tiie uni^t. 

X* Personal assessment of needs to stu_dy the unit . In this step, you 
first s^houl^ leaf quickly through the unit to familiarize yourself with the 
objectives and their content. Do this in a cursory fashion; it serves only 
* to set your mind in relation to the unit. Fifteen minutes should be enough 
time for skimming over the unit contents. 

Next, complete the Pre-Assessment Exercise (pages 5-7) as a-basis for 
estimating your present degree of mastery of unit objectives* The exercise 
calls upon you to think through each, of a series of steps in arriving at a 
plan to remeoy shortcomings- in implementation and to write down your 
tniustrative answer tor each st^p* You should not take more than 20-30 
minutes to do the Pre -Assessment Fxercise* 

When you complete the exercise, you should turn to page 8 for directions, 
on checking your answers with use of the Pre-Assessment Exercise Ansv/er Key* 
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PRE -ASSESSMENT EXERCISE - UNIT 9 ^ ^ ^ 

■ Planning to Remedy Shortconnngs in a. Program's Implemegtation 

/Directfons : This exercise offers you a way of xhecMtig how well prepared you. 
are to perform the steps 4n planning aTemedy for shortcomings in the 
impleineRtat^rr of a local educational program, This exercise shoold take 
mre. than one-half hour. Ohen you complqte the exercise, compare your 
answers to those gTven fn the Answer Key as explained on page 8. 

1. From the following description, write down a list of the features of the 
HIGS pro.gram to he implemeflteti^ in the Franklin Intermediate School. 

Methods of Inquiry in General Science (HIGS) is a new commercial science 
course for Intermediate schools. The two-semester course contains 100 . 
separate project units, each consisting of a study guide, a set of reading 
booklets, and a list of materials, equipment^ etc. to be provided by the 
school. There, are 20 projects each in mechanics, electricity, chemistny, 
plants, .and animals. Each student receives a copy of the Inquiry Methods 
Handbook as a guiti^ in planning projects' and preparing project reports. 
The science lab is to be divided into 5 learning stations, one for each 
of .the project areas, and a library center housing reading materials. Each 
learning station is to be equipped with what is required to perform the 
projects in its area and is to provide work space for 10 students, while 
the library center is to provide study space for 15 students. {These figures 
assume a class of 40 students or less,) . < * 

During each semester, each student is required to do One project in each 
area, two or three indepei^ently , and the others with a fellow student. 
Each student can do as many additional projects as Jime permits. The student 
elects each project with the instructor's approval. Not more than one stu- 
dent or student pair can work on a given project at a given time. The 
instructor assists students individually or in pairs, students present oral 
or vjritten reports on all *of their projects as the instructor's main basis 
' for evaluating their work.*; Both instructor and student keep an up-to-date 
record of projects perfonned and their level of accomplishment. 

list of program features {Be brief* You may or may not need all the numbers.) 

I. 

^ 2. 

3. 

4^ ^ 

5. 

6. . ^ 

7. 

8. 

9. ^ 

10. 

II. 

12. . 



13. 



r 
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•2(.'How vjould you ohtaiilf data on whether the Franklin I.S. chose^to modify 
tts version of MIGS tn any v/ay froiD what has beert described? 



3, y^ssume 3 science teachers each ha^ four classes in HLGS, What plan would 

you use to assess how v/ell the following procedures were employed in the*^ * 

12 classes? {Assume that you gather your data just after a semester has 
ended.) ' ; 

a- Students elect projects in conference with the'instructor,^ 

r 

\ ■ ■ 



f). E^ch student does projects n'n each area e^ch semester, two or three 
individually and the others, with another s^dent. 



i 

4. List aspects of the classroom setv^) for HIGS where it is likely that 
shortcomings of implementatijon'will be foUnd. 
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Assume that; Teacher A made extensive use of whole-class lectures and 
dts^ssions ''rather than teaching MI6S on an Individual basis* What are * 
a nupiber of likefy causes of this faulty implementation and how iifould you 
get data on tjjem? ^ 



S 



Assume you found tliat one major cause of Teacher A's failure to follow 
the required procedure was a lack, of understaiiding of \\m to individual- 
ize instruction- . Iffiat are types^of sources you would turn to in seeking 
information on v/ays, of overcoming this failure? ^ 



Outline a plan for remedying the shortcoming in Tead^ef A's Instruction 
and state reasons fdr your cTiotce* 



(Continue on back if you need more space,) 
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Now ttiaf you have completed the Pre-Assessment Exerctse, you'(and your 
instructor, if you have one) should canpare your answer:* with those given in 
the Pre-Assessineat Exercise Answer Key (end of unit). Compare the quality 
and detail of your answers with those in the Ansvjer Key. Yhere is no one 
right answer to any of the questions, /(iso, the A<^swer Key gives fuller 
answers to some of the seven parts of the Exercise than you would be expected 
to give. 

In the table below, check the .estimates you make of your degree of mastery 
of each part 0/ the Pre-Asses^ent Exercise. Check HIGH if you judge your 
answer to be right on target and in adequate detail. Check MODERATE if you 
judge your answer to be good but lacking some points needed for a ft'lly 
adequate answer. Check LOW if you find your answer to be inappropriate or 
very incomplete. t. 



Item # 



Stage in Planning Process 



Level of Mastery Sho rn 
High Mode1^ate How 



1. Listing key features of the MIGS program 

. 2. Finding whether Franklin I.S. made changes 

. 3. Plans for assessing program implementatiOi) ^ 

4. Listing likely shortcomings of implementation _ 

5. Li*sting likely causes of faulty implementation 

64 Finding likely ways to overcome shortcomings 

7. Outlining a plan chosen to remedy shortcomings 

Now, in the table below listirig the objectives of Unit 9, indicate for 

each whether you judge that you need only to study It for review purposes or to 

study it carefully. The item numbers" in the P re-Assessment Exercise related to 

each unit objective are indicated. 
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Identify and list essential features of a 

local educational improvement program (Items 1,2) 

Design a plan for assessing implementation of 

a program*s features Citems 3,4) . 

Design a v/ay of conducting a causal analysis of 

shortcomings in program implementation (Item 5) 

Survey likely ways of overcoming shortcomings 

in. implementation (Item 6) ' 

Design a plan toMmprove program implementation 



2. Study procedure . To study the unit, keep in mind that you will gain 
by doing the objectives in the order in which they appear since each section 
of the unit assumes a certain level of understanding based on the previous 
one- Some parts, according to your mastery estimates, will require your 
careful study while others will need only to be reviewed. 

You may wish to study all or part of this unit with one or more other 
students. Also, your instructor may elect to conducl^ group sessions* And, 
of course, you could study the unit entirely independ'ently- 

ProbaWy you will need one or two days to study this unit, depending 
on how intensively you v/ish to study any or all of the objectives* It is 
best to proceed through the unit as it is presented and then to make plans 
for later, more detailed study of those areas of particular interest to you* 

(when you complete study of the unit, you will find directions for the 
Post-Assessment Exercise and the Unit Evaluation*) 
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Objective 1. Identify and list the essential features of a local educational 
program from a v/ritten or oral description of it. 

In setttng about to improve the ImplementatTon of any program or program 
element of a school system, the first task ts to make a careful assessment of 
present shortcomings of implementation. This divides Into two sub-tasks: 
specifying the essential features of the program that need to be implemented 
and assessing the extent to which each of those features has been Implemented-' 
.Objective 1 focuses on the task of arriving at an accurate list of program , 
features. 

The essential features of a progi*am are those features that must be fully 
implemented in order for the program to be operating effectively . They are 
required rather than optional- They are elements in the blueprint of the 
program- They include th& "what" of the program {materials, equipment, 
arrangements, etc.) and the "how" {procedures to be followed)* They are the 
things one would need to examine in detemining whether or not the program had 
been implemented fully. 

Your leadership in arriving at a full and accurate list of essential pro- 
gram features is necessary, whether you are the program's director, a school 
system official with the responsibility of monitoring the program, or an outside 
consultant assigned the job of helping improve the program's implementation. 
Whatever your role, the task is the same, though the circumstances of your 
involvement will differ with the role, you are filling. The following deals 
with the process of obtaining a list of program features without regard to the 
particular role you are filling. 

^ A very frequent situation is when a school system has adopted a program 
developed outside the system. In such a case, it almost always is true that 
the prograpii^l^yfilopt?^^ written materials giving a description of the 

programVs features. Developing a list of these features then is simply a matter 

It 



Unit 9-11 

of abstracting them from the descrtptiort* 

Frequently a school system adopts some program develop'^d elsewhere 
and, in adapting ft to the local sftuation^.niakes certain changes in the 
program/ In thts case, tdentffytng program features calls for preparing a 
list from descriptions given by the originators of the program, then modifying 
that list from written or oral reports specifying the changes made by the 
school system. 

Many times school systems build their own programs, ordinarily selecting 
some features from programs developed elsewhere, and adding some features of 
their own, thus arriving at a program designed to fit the needs and character-* 
istics of the system. Developing an accurate list of features of such a program 
can be difficult since It Is not unconanon for a school system to have no formal 
written description of the program it created. Also It is not uncommon for 
changes to be made from the Initial written description of a program without 
their being written down in a revision of the initial description. In this 
case. It is necessary to interview those responsible for the program to learn 
just what changes have been made. 

To give you practice in listing program features. Exercise 1 calls upon 
you to identify and write down the essential features of a local educational 
program. The rule to follow in preparing your list is to identify every aspect 
of the program that you would expect to find in a school that implemented It. 

Exercise 1 

In this exercise, you are asked to prepare a list of features of a local 
improvement program from a written description of It and to indicate where in 
that description each feature is stated. The program described is that for an 
Intermediate school in a large metropolitan school district* 
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Use the Exercise 1 - Worksheet (page 16) in listing program features. 
When you locate one, make an abbreviated statement of it on the worksheet 
and indicate the paragraph(s) where it was stated. Here is an example of 
the sort of entry you should make* 

FEATURE DESCRIPTION OF FEATURE PARAGRAPH 

(A>B>etcO REFERENCE 

A* Use with grades 5-8 or 6-8 2 

(When you complete the exercise, check your entries with the Exercise 1 
Answer Key on P^ge 170 
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The (JtldwQod IntermedUte School Program (l-ftSP) fn ^ Metropol itan District 

Para- Considering the emphases and directions to be sought in the 

graph 

intermediate years of a child's education, WISP provides a nev/ school 

1 organization for use In achieving basic changes in curriculum, methodology, 
and guidance so that a more effective program of education can result. 

Although the direction is toward a four-year form of organization, 
some pilot schools wiP be made up of grades 6-7-8* Priority will be 

2 given to establishing 5-6-7-8 schools where such organization providtjs 
space in elementary schools for the expansion of the early childhood 
programs* 

WISP will offer enough variety and depth of Instructional material 
for most children to require four full years of attendance. Individual 
programming, hov^ever, will permit children to progress at different rates 

3 in their various subjects. Subject acceleration, resulting in advanced 
placement, will be emphasized. Thus, some children will be able to com- 
plete the regular sequence in three years, while other children will re- 
quire five years. However, fifteen-year olds, regardless of achievement 
levels, should be transferred to high school. 

Small schools-within-schools, each representing a cross section of 
the total school population, are to be set up. For example, in a school 
acconimodating 1800 pupils, three sub-schools of approximately 600 pupils 

4 will be organized. The 600 jjupils who belong to any one sub-school will 
be in grades 5-8 and will represent the entire gamut of age, ability and 
talent. These 600 pupils will be divided further into four blocks of 
about 150 pupils. 

17 
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WISP wtn program puptls for subjects such as art, mustc, health 
education and industrtal i(rts on a total -school basis, drawtng students 
froti) each sub-school* Common facilities, such as audttoriuiDs, gym- 
nastums, and musTc rooms, will be used jotntly, providing opportunttie^s 
for pupils to mingle with those form the other sub-schools in the 
building. In addition, it is recommended that pupils leave the sub- 
school for clinical help in reading, speech improvement, and other 
special services* 

For work.in other areas (reading, spelling, writing, mathematics, 
social studies, etc.)> pupils will be programmed for large- or small- 
group work in each sub-school or for working alone on activities suited 
to individual needs, WISP also programs each student for time in which 
to pursue individual interests. In order to meet individual needs. It is 
recommended that the conventional method of estimating the time needed 
by pupils to learn a given unit of work be changed. 

Basic to the WISP design is the diagnosis of pupils' needs 
and the individualization of instruction. To accomplish this, a team 
approach is required, calling for teachers, counselors, and other 
' specialists to work with administrators and supervisors in planning for 
the total educational growth of individual pupils for the duration of 
their stay in WISP. Each team will be assigned a block of approximately 
150 pupils and will be responsible for the progress of this group* In 
general, students will stay with the assigned team, although changes 
may be made if desirable. 
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The classroom teachers will be the school's basic guidance workers 
because they will have the closest and most continuing relationship with ^ 

8 the^puplls In their assigned group* Time blocks In the school day will 
enable teachers to confer with children, parents, and other staff 
members; to study records; and to plan ways for meeting their students' 
needs* 

V 

Based on a Pre-Kindergarten-Grade 12 sequence^ the curriculum will 
Include the commonly-accepted subject areiis: language arts, mathematics, 

9 science, social sciences, foreign languages, physical education, and the 
arts* In addition, typewriting will be taught In the fifth grade or as 
early as Is practicable* Foreign languages will be Introduced to all 
children at the fifth grade level* 

The coniplexlty of the WISP organization, with Individual scheduling 
and flexible progr?.ms^ will necessitate extensive supervision* Each 
10 school should have a principal, vice principal, assistants to the 

principal (one for each sub-school), and chairpersons in selected subject 
areas* 

The maxlmuni use of non-professional personnel (school aides, 
n lunchroom workers^ etc.) should be devoted to freetng the professional 
staff so that It can spend all of its time on professional activities. 
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EXERCISE 1 - WORKSHEET 

Listing Features of a Local Educational Program 

Oirections : Write iovm a list of the essential features of the Wildwood 
Intermediate School Program from the written description given. Make 
your descriptions of program features as bri'ef as you can without losing 
any key points. Be sur^ to give paragraph references. 



FEATURE" nFcrDTDTTrtM nF fcatiidf PARAGfyVPH 

A.B.etc. DESCRIPTION OF FEATURE REFERENCE 
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EXERCISE 1 - ANSWER KEY 
Listing Features of a Local Educatlona 1 Prog r am 



FEATURE nccrDTDTTfiM ftP FFJiTirPF PARAGRAPH 

A.B.etc, DESCRIPTION Or FEATURE REFERENCE 

At Use with grades 5-8 or 6-.. 2 

The student proceeds at his om pace through the 
program. 3 

C. Fifteen-year-olds go on to high school, regardless 

of level of achievement reached. 3 

D. The total school population is broken into separate 
schools-within-a-school (the house plan). 4 

Each sub-school has four age units of an approxi- 
mately equal number of students. 4 

F. Music, art, health, industrial arts, and special 
education draw students from all sub-schools. 5 

G. Both large- and small-group instruction occurs in 

all regular subject-matter areas. x 6 

H. Provisions for independent study are built into 

the program. 6 

I. A flexible time schedule is provided for work in 

any curriculum area. 6 

0. A staff team of teachers, counselor, other 

specialists and administrators is assigned to a 

team of 150 students. 7 

Teacher planning and conference tttriG tiS scheduled^ 8 

L. The curriculum is sequential and includes the 

regular subject areas. 9 

M. Typing and foreign language are Introduced tn the 

fifth grade. 9 

Each school has a principal, vice principal, 

assistant to the principal, and chairmen of 

subject areas. 10 

0* Paraprofesslonals are employed. 11 
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In listing prograiu features. It Is luipoHant to be sure that you 
have included both "structural** and **proceduraV' f^satures. any program, 
the structural features are those elements tSia^ form the framework of the 
program, and any materials or equipment included in ic* Thus, when a new 
curriculum Is adopted. Its structural elenvxt^ consist of texts or other 
sorts of learning materials, instructors* guides, equipment to be used, 
classroo:^ arrangements for Instruction, assignment of staff for teaching the 
program, and grouping and scheduling students for Instruction. All of these 
features can be purchased or arranged by those responsible for installing the 
curriculum. 

The procedural features of a program refer to things that staff members* 

students, or others are to do in conducting £he program. Thus, a new 
t 

curriculum may call upon teachers to plan students*' work on an individual 
basis*, to have students conduct their study In tGrms of individual projects, 
and to requir. students to demonstrate masiery of their learning tasks. 
These procedures are as much a part of the program as Its struct</^l elements* 
Usually they are the critical elements in implan-^nting a program and it 
usually Is here that faulty implementation occurs. 

Example of faulty implementation of procedural features within the 
structure of nongraded programs have been reported by GoodUd and Anderson, 
They found In a survey of nongraded schools across the country that many of - 
the elementary schools studied had set up groups on a non-grade-level basis 
but failed to teach in ways that enabled students to advance at different 
rates without regard to the usual grade-level curriculum. Within a nongraded 
structure, the teachers were teaching and advanring students according to the 
traditional grade system, (Goodlad, -lohn 1., and Robert Anderson. 
'^Educational Practices in Nongraded Schools; A Survey of Perceptions/* 
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Elementary School Journal , 1962, 43, 33-44.) 

To help you clarify the distinction between structural and procedural 

I 

features of a program. Exercise 2 follows. <^ 
Exercise 2 . 

\ In this exercise, your tas^ is to exainine the list of features Of the 
Wiljjwood Intermediate School Program as. given on the Exercise 1 - Answer 
Key* jDn the Exercise 2 - Worksheet, opposite each feature listecT check 
whether, in your judgment, it refers' to a program element that can be 
installed merely by administrative decisions or whether it calls for staff 
members'to do certain things within the structure provided. (Don't be - 
surprised if the majority of your check marks are in one column.) 

When you complete the exercise, turn to the Exercise 2 Answer Key 
(page 21) and compare your answers with those given. 



23 



Unit 9-20 

EXERCISE 2 - WORKSHEET 

Identifying Structural and py^ocedural Feature? of a Program 

Dtrecttons : Essential features of WISP are listed below. Check opposite 
each to Indicate whether, in your judgment, the statement Is of a 
structural or procedural feature of the prograiR. 



FEATURE STRUCTURAL PROCEDURAL 

A* Use the program with grades 5-8 or 6-8 

B. Students proceed at their own pace 

through the program 

C. IS^yeai'-olds go to high school, regardless 

of level of achievement reached 

D. Total school population Is broken Into 

separate schools*w1th1n-school 

E. Each sub*school has four age units of an 

approximately equal number of students 

Music, art, health. Industrial arts, and 
- special education draw from all sub-schools * x 

Both large- and small-group Instruction 

occur In all regular subject areas t 

H. Provisions for Independent study are 

built into the program 

I- A flexible time schedule is provided 

for work in any curriculum area 

0. A staff team of teachers, counselors, 
other specialists and administrators Is 

assigned. to a block of 150 students 

Teacher planning and conference time 

Is scheduled ^ 

Curriculum Is sequential and Includes 

regular subject areas 

M* Typing and foreign language are 

Introduced In the fifth grade 

Each school has a principal, vice 
principal, assistant to principal 

and chairpersons of subject areas 

0. Paraprofesstonals are employed 
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EXERCISE 2 - mSHZR KEY 

Identifying Structural and Procedural Features of a P rogram 

In this answer key, the check marks distinguish structural and 
procedural program elements by assigning as structural those elements that 
involve primarily administrative actions and by assigning as procedural those 
elements that specify what needs to be done within the program's structure. 

FEATURE STRUCTURAL. PROCEDURAL 

A. Use the program with grades 5-8 or 6-8 , ^ ^/ 

B* Students proceed at their own pace 

through the program ^ 

C. 15-year-olds go to high school, regardless . 
-of level of achievement reached ^ 

D. Total school oopulation is broken into ' . 

Separate schools-within-school , y 



Each sub-school has four age units of an 
approximately equal number of students 

Music, art, health, industrial arts, and 
special education draw from all sub-schools 

Both large- and small-group instruction 
occur in all regular subject areas 

H. Provisions for independent study are 
built into the program 

L A flexible time schedule is provided 
for work in any curriculum area 

0. A staff teaiD of teachers, counselors, 
other specialists and administrators is 
assigned to a block of 150 students 

Teacher planning and conference time 
* is scheduled 

L* Curriculum is sequential and includes 
regular subject areas 

H. Typing and foreign language are 
introduced in the fifth grade 

Each school has a principal, vice 
principal, assistant to principal 
and chairpersons of subject areas 



V. 



Paraprofessionals are employed { ^ 
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/ It often happens that a school system leader or external consultant: * 

. needs to develop a list of features of an ongoing school system program 
without prior acquaintance with that program. Starting from scratch, how 
should one proceed? The following procedure Is offered as onie way of getting 
the needed Information. ' 

1. Contact local personnel to detemlne who Is responsible f or^ the 
program and v/here Information about It can be found. 

Z, Obtain written materials descrlbinn the program, if they ex;st. If ^ 
the program vas developed outside the school system. It Is likely that there 
are-detailed descriptions of Its features available In writing. If it was 
created v/lthin the school syston, there may or may not 56 a written' 
description available. For example. If the elementary reading program Is the 
area of concern^ It Is apt to be true that It has evolved oveWHhe years with 
changes In materials and procedures that have never been formally described in 
writing. Assuming some v/rltten description of the program Is available, a 
first step Is to prepare a trial list of program features from the 
descr'fptlon, 

3. Interview program participants - supervisors, teachers, or others - 
to develop an Initial list of program features If no written description Is 
available, or to round out your Initial list derived from a written 
description. In conducting these interviews, it is especially Important to 
probe for pro^:edural features of the program that specify wnat staff members 
are to do In condlicting the program. An example of this task can be^ found in 
the description of the Wlldv/ood Intermediate School Program used in Exercises 
1 and 2, If you turn to the Exercise 2 Answer Key, you will note that some 
of the structural features imply Instructional procedures that are not spelled 
out. For Instance, Feature 0 states that "paraprofessionals are employed" 
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but, says nothing abou^what they are to do. Do they prepare learning 

materials? Check students' work? Keep records? Etc. One of the procedural 

features on the WISP list (Feature B) says that students proceed at their own 

pace through th^ program without indicating how this is to be implemented* 

How are teachers to determine where individual students are in the learning 

sequence? How a^re they to p'^an individual lessons for them? How are they to 

arrange for them to complete lessons at different rates, and to permit them 

to move on to the next task without waiting for other students to catch up? 

Procedural features such as these need to be included in the list of essential 

features of a program. 

4* Edit your initial list of prograin features to satisfy the following 

criteria: ' ^ ' 

All features on .the list should be essential elements of the program 
■ that must^e implemented for t\^B program to be in fuTl operation* 

The list shoula include^ both structural and procedural features/ - * 

Statements of tie features must be expressed in a way that indicates ^ 
what sort of evidence is needed.to determine their implementatiort. 

5. Check your edited list by re-int^jrviewing the program supervisor and 

participants to make sure that no essential elements have been left out* 

Your final list mu§t include a full set of program features since it provides 

the needed basis for ^lU of the succeeding steps in improving the program's. 

impljsmentation*" Thus, in the case of the list of , WISP features, several would 

need to be added -to specify procedural aspects of the program not included in 

the written description of that program. 
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Objective 2* Design a plan for assessing the implementation of the 
features of an educational program* 



Assume that an accurate list of the essential features of a program 
has been made* The task now becomes one of preparing a plan for assessing 
the extent to which each of these features is in operation in the schools 
conducting the program* The purpose of this assessment is to determine 
which features are not well implemented* A discrepancy between the feature 
as specified in the program design and that feature as implemented identifies 
a need for improving the program's implementation* The diagram below 
represents the difference that be found betv;een the full implementation 
of a feature that is intended in the program's design and the actual level 
of fts implementation* 



In most cases, the assessment plan should be selective or diagnostic, 
rather than exhaustive, because its purpose is to identify major shortcomings 



in the irtiplementation of program features* Highly sophisticated assessment 
methods, lengthy examination of the operations of a program, W a study of 
implementation in every classroom vjhere the program is being conducted, usually 
are ''not feasible* 



LEVEL OF COMPLETE IMPLEMENTATION 

Need for improvement 

ACTUAL LEVEL np IMPLEMENTATION 




LEVEL OF ZERO IMPLEMENTATION 



r 
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You would need to do several things In designing a plan to assess 
a program's Implementation. One Is that you choose appropriate methods 
for obtaining Implementation data on each of the program's features. Another 
Is that you examine the resources available for conducting the assessment- 
time, funds, expertise In assessment methods, assistance from school district 
personnel, etc. Yet anothep^ls to determine the scope and duration of the 
assessment effort—how many schools or classes can be involved In the 
assessment and over v;hat period of time is it to be conducted? 

Objective 2 will help you deal with these concerns* It Is organized 
Into two steps: 

Identifying methods for obtaining data on the implementation of 
program features. 

Setting up a plan for assessing the Implementation of program features* 
Several examples and exercises are Included to provide you with models for 
consideration and practice opportunities. 

Identifying Methods for Obtaining Data on 
the Implementation of Program Features 

Five general methods for obtaining data on the Implementation of program 

features are available to you: (A) interview, (B) questionnaire or Inventory, 

(C) analysis of diary or journal entries, (D) analysis of records or reports, 

and (E) observation* It Is not a purpose of this unit to teach basic skills 

In the use of these methods; that v^ork requires a longer time and should be 

met by course work In evaluation methods. It suffices In this unit to examine 

the value and limitations of each method for obtaining data on different 

types of features of local educational programs". 

A* Interview ^ Interviewing key school district personnel who are 
associated with the program to be studied is a first step In obtaining d^ta 
about the implementation of the program. The local program administrator or 
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coordinator can give you basic data about the origin, design, and scope of 
the program* Often you can use interviewing to ascertain the implementation 
of general structural features of a program. In addition, interviews with the 
building principals or with teachers may yield at least preliminary evidence 
on the level of implementation of the procedural elements of a^ogram* 

Generally, though, interviewing is insufficient because those conducting 
a program, even classroom teachers, are apt not to have accurate information 
on how fully the specific procedures of a program are being carried out* 
Teachers may be too close to have an overall program picture, while 
coordinators or. administrators, having the broad perspective, may lack 
familiarity with classroom occurrances* Also, participants in a program 
tend to give a more favorable picture of program implementation than is 
justified* One way of correcting for such inaccuracies in the interview 
approach is to request specific data on what happened today or di^ring the past 
few days on the assumption that this will tend to yield more objective data 
than a more general inquiry* Another corrective is to ask a teacher to report 
what occurred with a particular child and to exhibit the child's work or 
records of work at the same tim. 

An obvious advantage of the interview approach is that it is not limited 
to the immediate present—as U observation* Also, when the program 
investigated is not focused in instruction, it will often be necessary to rely 
mainly on interviews because there is less to observe directly* This is true, 
for example, of educational programs in school -community relations where 
opportunities to observe program features in action may be quite limited* 

B. Questionnai re > A substitute for, or supplra'^nt to, the int^erview 
approach is the use of questionnaires or inventories calling for written 
responses to items or questions* The questionnaire approach does not require 
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one to be present and so can be administered to any number of people 
simultaneously* Thus, this method might be used to obtain data from every 
teacher in a program or from parents in the community. Also, respondents can 
fill it out at their convenience* With this approach, it is possible to obtain 
at least preliminary data on the implementation of any feature of a change 
program* 

However, the same limitations that apply to interviewing apply more 
strongly to the questionnaire approach* There is the dangerrthat answers to 
questionnaire items, whether the respondent checks an alternative or writes 
in an answer, will be inaccurate due either to lack of specific knowledge or 
a tendency to present a favorable picture of the program* In addition, there 
may be a low rate of response or slowness In responding* Finally, no visual 
cues are present to aid the administrator of the qu^esMonnaire In deteminlng 
the validity of the responses* 

C. Diary or journal * Another method for obtaining data from others is to 
induce them to keep a diary or journal In wh^ich they record, daily or weekly ^ 
the occurrence of activities within a program. If participants in a program 
are willing to subject themselves to the discipline of keeping an accurate 
journal of activities, this can provide valuable data on the implementation of 
program features* 

It is difficult. to persuade busy participants to keep a regular journal 
on specific activities they or their students perform, since it represents one 
more, possibly annoying, task to carry out. Even when they do keep such a 
record, it usually needs to be followed up or supplemented by data obtained by 
other methods because the tendency is to make journal entries brief and 
cryptic rather than explicit. 
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D. Analysis of records or reports . For obtaining data on some program 
features, routine records of a school system m^ prove to be valuable. Class 
rosters can provide data on the characteristics of students making up class 
groups: age, sex, grade level, reading level. Teachers' manuals indicate 
what the teacher is expected to do in the classroom. Reports from when the 
program was put into operation can offer insight into what was considered 
important at that time and shed light upon probable causes of difficvilty 
being experienced now. Sometimes it is necessary to combine data stored in 
different places, e.g., information from class rosters can supplement data 
from individual student folders to provide a fuller picture of the population 
participating in the program. 

In cashes where studies have been conducted of the implementation of a 
program, reports of such efforts can provide a valtrable shortcut to some 
aspects of assessing the Implementation of a program. Always, however, there 
is the danger that the study was inadequate or the report inaccurate. 
Generally, any findings of such studies should -be checked against data freshly 
obtained. 

E. Observation . For numerous features of a local educational program, 
on-the-spot observation is the critical method for obtaining data on levels 
of implementation. This is particularly true for assessing the implementation 
of the procedures that staff members and students, or others, are to follow in 
operationalizing the program. 

Observation has the advantage of providing firsthand evidence, since 
you can observe events as they occur without relying on the report of an 
intermediary. But, several hazards limit the value of observational data. 
One concerns the correctness of the categories of behavior observed: Have the 
features of the program been reflected clearly in decisions as to what types of 
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observational data to seek? A second hazard deals with the appropriateness 
of the units of observation and whether they are measures of specific acts 
or ratings of events that occurred* Another problem concerns the question 
of sampling. How can one be sure that, with limited personnel and time, the 
best set of observations to be made has been determined (involving class and 
teacher differences, time of day, etc.)? A fourth hazard concerns data 
analysis and Interpretation. Not only should methods of analyzing data be 
carefully chosen, but the evaluator must carefully guard against being overly 
optimistic or pessimistic in Interpreting data obtained by observation. 

In summary, consider the following: 

1) Choosing the data-gathering methods to use In determining the degree 
of implementation of features of a local educational program is a vital part of 
the strategy for assessing program Implementation. Generally speaking, for any 
given program feature, one of the five methods outlined should be chosen as the 
anchor method for the accuracy and completeness of the data it can provide. 

2) Interviews, perhaps supported by questionnaire or diary records, have 
their special functions In obtaining data on program implementation* The 
Interview normally is the a.ichor method In obtaining orientation data about a 
local educational program, on the Implementatlori of structural features of the 
program, and clarifying data obtained by other means* Where Information from 
many people is needed, the questionnaire approach often must be used because 
of lack of resources for conductli.^ Interviews with everyone Involved in the 
program being studied. 

3) When applicable and feasible, observation should be relied upon as the 
anchor method for determing the actual Implementation of progran features. 
Observation— seeing and hearing— can offer the most direct and incontrovertible 
evidence of implementation. Closely related to observation is the analysis of 
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school records^ Schedules, class rosters, reports, t,est data, and student 
folders are valuable sources, although they frequently are incomplete, 
. inaccurate, or both* 

Exercise 3 

To give you practice in choosing data-gathering methods, this exercise 
presents you with a list of the features of the Achievement Competence Training 
(ACT) program and asks you to identify rnethods appropriate for obtaining data 
on each feature. 

Before doing the exercise on the Worksheet, read the brief description 
of the program* Then turn to the Exercise 3 - Worksheet where you will find 
the ACT features .listed* For each feature, check those of the five methods 
that v/ould give you valuable data on that feature* Then, for the same 
feature, circle the check mark for the method that, in your judgment, can 
provide the most reliable and complete information on implementation of the 
feature* Do not hesitate to give your opinion in cases where you are uncertain 
as to_ which methods to check and/or which check mark to circle* 

When you have completed the Worksheet, tui^n to the Exercise 3 - Answer 
Key on page 34 and compare your marks with those given there. Remember> 
the purpose of this exercise is simply to offer practice in identifying 
appropriate data-gathering methods* There are no official right or wrong 
answers* 
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The flchievement Competence Training (ACT) Program 

Achievement Competence Training (ACT) is a prograih Intended to assist 
learners In becoming self-directed* By treating children as active agents 
In their own development, ACT aims to promote a vital sense of Internal 
control and self-esteem which is closely related to effective learning* ACT 
seeks to teach precise goal-setting and planning skills which research has 
linked v/ith achievement behavior; ACT structures exper^iences to aid children 
in using those skills autonomously* Students are encouraged to transfer ACT 
skills to all other school activities and to activities outside school; 

The. ACT program is designed for use with upper elementary school children 
and is suitable for use with grades 5, 6, and 7* The Instructional package 
currently available has been developed and refined in a fifth-grade setting* 
The ACT materials require^ class mean grade-level reading score of 3*5 or 
higher* They have been designed to be culture, color, and gender free- 

ACT consists, of a sequential Order of 4 units, divided into six parts 
each* Each unit builds upon the previous one, extending learner application 
of the six achievement steps; (1) self-study, (2) get goal ideas, (3) set 
a goal, (4) plan, (5) strive, and (6) evaluate- 

The ACT package is programed on audio-tape and is designed to be used for 
three 45-minute periods per week for a semester* Parts are structured into 
lessons that can be used in 15-20 minute segments if the teacher prefers 
shorter periods- 

ACT lessons are presented in a multi-media format, incorporating audio- 
tape cassettes, printed student journals, filmstrips, and games* Taped 
information, music, and dramatizations guide learners through the highly 
Illustrated student journals* The journals serve as a record of personal 
information which the children use to set goals* The ACT materials then teach 
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a process whicK ea'afales children* to achieve these sa?ie goals. In a sense the 
interests 'Bf th^. Children are, a major part of the contertt of the materials* 
Additional materials not supplied but required are an audio-tape cassette 
player, a filmstrip projector, regular elementary school art and construction 
material, and a room equipped with movable furniture- 
Each part of ACT presents information to the children, calls upon the 
student to engage in experiential activities, and requests application of 
information learned to real-life activities. Typically, students listen to a 
tape that presents concepts, ^engage in various activities using their own 
journals as directed by tape, and work individually or in small groups when 
the tape is turned off. Pretests and posttests are incorporated in the program 
at regular intervals* ^ 

Although the audio-tape leads the students through each ACT lesson, the 
teacher is called upon to assist in structuring many activities, to reinforce 
concepts, to assist individual children, and to correct some pages in the 
students' journals. The teachers' mnuals with annotated scripts for each 
lesson guide the teacher's participation and provide answer pages for journal 
exercises and tests. In addition, teachers can foster the transfer of ACT 
skills to other school activities and out-of-school activities by motivating 
students to use ACT skills outside the ACT program, and by reinforcing students* 
transfer of ACT skills. 
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EXERCISE 3 - WORKSHEET 



Data-gatherinq Methods for Achievement Cora 


petence Training {ACl 


') Prograra 


FEATURE 


INTER- 
VIEW 


QUESTION- 

M A T D C 

NAIKt 


DIARY > 
jouraiAL 


RECORDS, 

KEPUKTS 


08SER- 

1/ ATTflM 


A, Prograra is used only in 
grades 5, 6, or 7 












B, All students studying ACT 
should be at or above 3*5 
grade level in reading 












C* The 24 program parts should 
be presented in sequence 












The program covers 1/2 
year.at rate of 2-2 1/2 
hours per week 












E* Pretests and posttests are 
employed 












F* Multi-raedta materials ate 
used 












ACT instructions on conduc- 
ting unit activities are 
followed 












H* Students set goals and plan 
ACT lessons 












I, Students transfer ACT skills 
to other school activities 












0* Students transfer ACT skills 
to out of school activities 












K. Students keep a journal of 
their activities in ACT 












L* Students work individually 
or in small groups 












Teachers follow the manual 
for lessons 












rit 1 eacners assist s^uoen^s > 
correct journals, reinforce 
concepts 












0* Teachers guide students in 
transfering ACT skills to 
school activities 












P* Teachers guide students in 
transfering ACT skills to ■ 
out-of-school activities 
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Explanation of Answer Key 



You v/ill note that the ansv/er key checks all of the items A-P forr 
Interview and Questionnaire . But note also that the questionnaire^ approach 
is not circled for any item, indicating that it is not considered to be the 
most adequate approacfTto getting data on any feature of ACT, The questionnaire 
method would be used as the primary source of data only when a more valid method 
could not be used. Often, of course, the questionnaire method 1s used to 
supplement data obtained by other means, as when one wishes to obtain some data 
from a large numbei^ of sources (teachers, classrooms, etc.). 

The interview method has been, checked as the most valid source of data 
on five of the ACT features. Note that these are structural features of the 
program (A, C*E} or a feature that does not offer datt* from^ observation (J). 

Records or Reports is the column where the checkmarks ar^ circled for 
features B and K. These features should show up on class rosters (feature B) 
or student products (feature k). 

Observation is circled as the preferred method for those ACT features 
that would show up directly in Instruction, Obviously valuable data on these 
features can be obtained from interview and questionnaires. However, to avoid 
inaccuracies that are common with these methods, chief reliance should be 
placed on observation in a representative sample of situations. 

4f 
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Setting Up a Plan for Assessing the 
Implementation of Program Features 

Background Informatiori 

Assume that ySur task is to develop a plan f<^ assessing the implemen- 
tation of the features of a cooperative teacKing program in Bigtown Inter- 
mediate School (grades 5-8). The program has tKe following 11 features: 

1. Each team covers instruction in the. four major curriculim areas of 
English, social studies, science, and matKematics. 

2. There are 12 teams and each consists of four teachers and one full- 
time paraprofessionaK ' ' , 

^ 3. One teacher has the role of team leader with responsibilities for 
calling and cha.i**^r^5 team meetings, resolving problems of team 
operation, and -^epresenting^the team to the assistant principal. 

4. There are about 120 students in a team. 

5. Students in, a team cover two grades, 5th and 6th, 6th and 7tK, 
or 7th and 8th. ' 

6. Each teacher on the team advises abbut 30 students with responsibility 
for advising them in both basic and special subjects. 

7. Multi-disciplinary instruction is to be provided regularly by 
^integrating material from or more of the four basic subjects- 

8. Nongraded advancement is provided; a student can advance at any 
rate, fast or slow, tRrough the curriculum in each major subject. 

9. Weekly team planning permits varying the amount pf time different 
students spend on a subject and all ows teachers to v/ork with intiividual 
students, with small groups, or with class groups of 30-40 students. 

, 10. Each day, all students in the team are assigned to special subjects 
(art, music, shop, PE, etc.), during a two-hour block. of time that 
enables team members to attend team meetings, preoare lessons, etc. 

11- The aide prepares lesson materials, gi^ades objective exams, and 
monitors the use of learning materials and equipment. 

Bigtown Intej.nediate School has four houses, eacK containing about 360 

students covering grades 5-8, Each house has three teaching teams- The team 

program has been in operation for twc years- An assistant principal is in 

charge of the cooperative teacKing program at Bigtovm and has responsibility 



40' 



Unit 9 - 37 

for asstgntng staff men*ers and students to teams* for appotnttng team leaders, 
for the training given the teams, and for supervising team operations. 

She holds a weekly meeting with the 12 tetun leaders' 

Planning Tasks for Asgessing Implementatioti 

Your responsibility is to plan an assessment of the tmplenentation of 
each feature of the cooperative tesiching program. Your plan will deal with 
identifying shortcomings in the isiipl orientation of any of the features in any 
of the 12 teams/ but your chief purpose is to identify those shortcomings in 
implementation that occur in several, if not most, of the teams. 

Assume thai you have one month during whtch to obtain your data and 
that school is in session during that entire period* You have the 
full support of the assistant principal in charge of the program and the 
promise of cooperation from all of the team leaders. Any secretarial or 
clerical sef^vices that you need also will^be available to you. 

The planning that you will need to do can be broken into five sub-tasks, 
as follows: 

1. Specifying the purposes and scope of the assessment of Implementation. 

2. Obtaining a representative sample of data on program implementation* 

3. Choosing methods for getting data on implementation of program 
features* 

4* Setting up a data-gathering schedule. 

5* Planning how to analyze the implementation data> 

These five tasks now will be considered in turn* 

1. Specifying the purposes and scope of the assessment of implementation * 
Your overall objective in ^^sessing the implementation of the features of 
the cooperative teaching progr^ri* is to identtfy those features in which 
there are serious shoi tcomtngs in at least several of the twelve teams. 

Recognizing that you have Just one month to gather your data and that 
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you roust do the job largely alone, you will want to plan use of your time 
wisely so ^ to net detailed and relevant information. Much of your data 
will have to be obtained through interviewing people and through using 
questionnaires. Your time spent with each team will need to be limited to 
about one day of observation and one day of interviews with team members. 
Structural features of the program (1-6) can be investigated quickly with the 
use of questionnaires, and through interviews with the vice principal in 
charge of the program or team leaders. Using these methods for those areas 
will permit you to concentrate your time on the proced al features (7-11), 
whose implementation is harder to assess: 

2. Obtaining a representative sample of data on program implanehtation . 
The problem of sampling the data potentially available on program implementation 
arises since you cannot hope to obtain complete data on the operating levels of 
features where there is .variation from teacher to teacher and day to day. 

You can obtain essentially complete data, and quickly, about the 
structural aspects of the program because you can assi'me that, at least for 
the period of your investigation, these elements will remain unchanged. The 
problem of sampling, therefore, centers on the functioning of the program* 
rt is clear that you will not be able to obtain all the data you would like 
on any of these topics* Through interviews with team leaders in advance of 
observation, you can roake roore effective use of your time spent visiting a 
team since in this v/ay you can learn where and when to look for evidence on 
team procedures. Also, it will help you greatly in getting accurate data on 
tearo procedures if you have a tarn member at your elbow when observing the 
conduct of student advising and instruction^ 

3, Choosing methods for getting tmplomentation d5ta ^ Wtth the four 
team roles in mind, your job now is to decide which methods you will use to 
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gather data on the implementation of each feature of the cooperative teaching 
program* Remember, five types of investigation are available to you - 
interview, questionnaire, diary or journal, analysis of records or reports, 
and observation* Keep in mind that your time is limited, so you should choose 
those methods that are most economical of time and still capable of providing 
valid data* 

In choosing your data-gathering methods, a first step is to plan how 
you will obtain data on the structural features of each team* Implementation 
of the structural features can be checked quickly enough by (ireparing a 
questionnaire to be completed by each team leader, though you may need to 
supplement your questionnaire with an examination of team rosters r*nd 
schedules* 

To assess the performance of the four roles you need to have data on 
whether the functions are performed , whether they are performed regul arly , 
and whether they are performed well * Consider these questions in connection 
with each role* 

For data on team members' performance of the roles of team leader , 
student advisor , ins1:ructor , and p ara£>rofessiona ) , you may rely chiefly on 
having the twelve team leaders, the 48 teachers (including the team leaders), 
and the twelve paraprofessionals submit reports on samples of their role 
performances* Probably, reports are needed for each day during one week, 
and they should follow a standard set of topics or questions so that they 
cover essentia*! items and provide data fron different respondents that can 
be combined readily* These diary-type records should be supplemented, where 
appropriate, by exhibits of schedules or reports and by observatk and 
interview* 

Probably the best way of accjuiring data on team leadership \s to ask 

* 

43 



Unit 9 - 40 

team leaders to keep a daily di^ry record of whn they have done as team 
leaders during a period of one or two weeks* They should tncli|de in these 
accounts descriptions o^ any problems encountered in performing the role* You 
should specify the types of information requTred and offer a number of examples 
covering the spectrum of their duties< It helps considerably to provide 
report forms ^to the team leaders so that they can keep the task before them 
as they carry out their work* 

To round out the leadership picture, plan to get observational data by 
attending a team meeting of each team to determine if they are well-focused, 
orderly, efficient, and friendly^ More observational data on the smoothness 
of team functioning as menrfcers go aWit their individual tasks for the day will 
offer information from a slightly different perspective* Points of view other 
than that of each team leader (diary record) and you (observation) can come 
from questionnaire (or interview) evidence of team members attitudes on the 
effectiveness of team planning ;ind oi\ cooperation among team members* And 
finally, valuable data on the quality of team leadership can be acquired from 
the vice principal in charge of the program. 

Data on the perfomance of the advisory role can be obtained in several 
ways* First, a questionnaire to the 48 team teachers on their performance of 
the advisor's role Is recommended. The questions should elicit a factual 
report on the frequency, length, and content of individual and of group 
advisory sessions* To ensure greater accuracy In these reports, it may be 
desirable to observe what teachers do during the daily two-hour period when 
students are in special subjects* Are students called in for individual 
conferences? What takes place? You may want some more detailed information 
on the role and might consider asking teachers to present a diary record of 
advising for a one- or two-week periods Then too, interview data from team 
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leaders and the vtce principal should be sought for general infomation on the 
perfomance of the advisory functiorv. 

Exercise 4 gives you practice in conceptualizing and listing content 
areas of needed data on the role of advising students. When you have completed 
the exercise, turn to the Answer Key that follows to compare your ideas with 
those suggested there. 
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EXERCISE 4 - WORKSHEET 

Outline for Obtaining Data on Advising Students 

Directions : Assume that your purpose is to obtain data on how many advisory 
conferences a teacher has with individual students or groups, on the 
purposes of those conferences, on their content, on their duration, and on 
what effects the conferences have* What are questions'70lj would want 
answers to in a questionnaire to teachers on th§ir general performance 
of the advisory function, and topics you*d want teachers to report on 1n 
a diary record of one week*s advisory conferences? 

Questions in a questionnaire for team teachers on student advisory conferences 



Topics for a teacher's daily record on advisory conferences : 
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aERCtSE 4 ^ ANSWER KEX 

Outline for Obtaining Data on Advising Students 

In the list of features of the cooperative teaching program, it was 
specified that each team teacher would have about 30 students with the 
responsibility of advising them on progress and problems in both basic and 
special subjects. 

The following lists of suggestions are meant to h^p you check the , ' " 
questions or topics you listed on the Worksheet for Exercise 4*^ ^J^^ . 

Questions in a questionnaire for teachers on their student advisory conferences 

These are some aspects of the advisory function on which data would be 
valuable: 

Nuipber of advisory conferences held, on the average, during a week (or 
month) 

Per cent of these conferences held with groups, and groups of what size 

Per cent of conferences held with individual students and reasons for 
these conferences 

Frequency of conferences with any given student 
Length of conferences with students 
Content of conferences 

Ways of getting data from the student*s different teachers as a basis for 
total-program advising 

Decisions, pl^ns reached in conferences 

How decisions are reached and who has Input (students?) 

Teachers' views about the advisory program: strengths and weaknesses 



Topics for a teacher^s diary report on student conferences 

For each reporting day, list the student conferences held and, for each, 
report the following: 

With whom the conference was held 

Who initiated the conference ' 

Purpose of the conference 

Duration of the conference 

What was discussed 

How the conference was based on' information from the student's teachers 
Decisions reached at the conference 

Fol1ov/-up actions (reporting to other teachers or team leader; plans 
for future conferences, etc.) 
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In struct ton tn the cooperattva teaching program calls for a multi- 
disciplinary approach, nongraded advancement, and flexibility in scheduling 
and grouping. Tou will have to rely primarily upon observation to assess the 
manner and degree of implementation of these features- However, with each of 
them,^ou can get data on the general ways in which they are Implemented 
through interviewing the vice principal and team leaders. Also, valuable data 
can be obtained by examining weekly schedules planned during the team meetings, 
by requesting samples of lesson plans to check on multl -disciplinary 
instruction, and by asking for records which indicate that students In. the 
same grade are permitted to advance at different rates In the major subjects. 

The value of diary reports by team members on the conduct of Instruction 
must not be overlooked. In fact, obtaining such reports before you visit a 
team would be of great help to you In orienting your observations and your 
questions to team members. If the diary reports from all team members covered 
the same period of days, you could combine them to get a picture of Instruction 
for the whole team that would reveal, for example, how much flexibility in 
scheduling and grouping occurred. 

Since Instruction In a team varies considerably from day to day, it is 
Important that you try to visit a team on day when all of the major 
curriculum areas are covered and when there are i^o unusual events scheduled. 
You would have to check with the team lea.der a few days before you planned to 
visit in order to determine if that particular day was suitable. Having 
some open dates in your schedule offers considerable leeway in case it is 
necessary for you tcf rearrange your visit. 

When observing a day's instruction in a team, it is essential that you 
use standard procedures for observing and reporting. Otherwise, you cannot 
be certain that you are covering essential aspects of instruction or that your 
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data could be combined readily once obtained^ One observation method that 
facilitates easy^ collatfon ts the use of a checklist with questions pertaining 
to the program under study. The observer sft;iply checks the appropriate boxes 
during a particular time span, say, a class period^ and briefly notes the 
evidence leading to choices. This approach yields categorized data but, 
as might be expected, it does not provide the rich detail common to other 
methods, such as the running account, in which the observer writes down every- 
thing that happens within a given time period and keeps a time line in the 
Meft margin, in that method, each activity is described specifi^cally in 
terms of the lesson, the students, the teacher{s), and the aide{s). 

A difficulty in writing detailed reports of observations is that you 
cannot observe and write at the same instant. Also, it often is not desirable 
to be seen writing notes on your observations. Thus, you may need to find a 
private place several times a day for writing your most recent observations. 
This difficulty usually is manageable tn completing a checklist. 

Try Exercise 5 now, which calls for you to develop a list of questions 
thati:might be useful for an observation checklist. Remember that the questions 
must be stated in such a w^ as to elicit a yes/no response; for example; 

''Are pre-tests used to determine what parts of a unit a student 
needs to study?" 

You will note that there are three categories on the worksheet; 

1) Multi-disciplinary instruction 

2) Nongraded advancement 

3) Flexible scheduling and grouping 

Your questions should be pertinent to these categories. A brief review of the 
list of features of the cooperative teaching program may be helpful to you at 
this point. 
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After completinci your list of questions, turn to the Answer Key that 
follows to compare your checklist with the one suggested there* 
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EXERCISE 5 - WORKSHEET 

Developtng an Observation Checklist 

Directions : Under each of the three headings, write a list of questions 
for which you would seek answers when observing instruction by a team* 



Team Leader_ 
Teacher 



Student(s) Taught 



Grade level of team: ^-6 

SubjectCs) Taught; 

Date 



7-8 



Time Period 
Observer 



HULTI-DISCIPLINARY INSTRUCTION 


VES 


NO 


EVIDENCE 










NONGRAPyED ADVANCEMENT 









FLEXIBLE SCHEDULING AND GROUPING 
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EXERCISE 5 - ANSWER KEY 

Developing an Observation Checklist 

Explanation ; The following are reconmended questions. Doubtless you have 
thought of other valuable ones. 

HULTI-DISCIPLINARY INSTRUCTION 

K Are materials from two or more of the four basic subjects integrated? 
2. Does this integration occur fairly frequently? 

3* Does the integration appear to be meaningful and relevant to the lesson at 
hand? 

4. Does the student appear to be confused at the integration of two content 
areas? 

5. Do materials lend themselves to integration? 
NONGRADED ADVANCEMENT 

1. Are stcdents froni different grade levels studying same lessons? 

2, Are students from the same grade level working on tasks at different 
levels of advancement? 

3* Are students in lower grade level of team studying at a level below usual 
for their grade? 

4. Are students In upper grade level of team studying at a level above usual 
for their grade? -.^-^ 

5. Does nongrading apply to Individual pupils? 

6. Does nongrading apply to groups of^pupils? 

7. Can pupil differentiate own level of wrk? 

FLEXIBLE SCHEDULING AND GROUPING 

K Do different students spend different amounts of time on a given 
subject? 

2, Do teachers work with individual students? 

3, Do teachers work with small groups? 

4* Can Individual students spend less {or more] time on one subject than on 
another? 

5. Can students work in groups of 2-3 to provide help to one another? 
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■ Finally, obtaining data on the paraprofesstonal function of the cooperativi 
teaching team also can employ several methods/" Asking each aide to keep a 
specific, detailed diary record for one or twp weetcs probably is the best way 
to get cogent infonnation on the perfonnance of this role* Once again, it is 
advisable to provide report forms (as was recommended with team leaders) so 
that the aides can keep this additional task in mirid as they carry out their 
regular duties. An example of such a report form is presented on the following 
page* Is there anything that you would add to it? 

In addition, interview data from the team leader on strengths and 
weaknesses in the aide's work should be obtained* Teachers *^ attitudes about 
the help that they receive from the aide should be explored as well. While ^ 
observing the team, you will be able^ ^et a notion of the aide's role 
perfonnance for a clearer, profile. 
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Your name 

Team leader's name 



Date this report is for 

Grade levels of team: 5-6 6-7 7-8^ 



Directions : Please indicate below, tn chronological order, the activities you* 
performed during the day being reported* Include such tasks as preparing 
materials, working with students, conferring with teachersy^ attending team 
meetings, and any others in which you engage* Be specific* When several 
activities occurred at once, please indicate what they were** For each 
activity or set of activities, indicate the time period in the left margin* 
Be as accurate as possible in noting the amount of time spent on an activity* 
You may continue on the back of this sheet and then go on to a second page if 
you need space* 



Time^ Period 




From - To 

I 


Activities 
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4, Setting up a data-gathering schedule . Setting up a schedule for 

collecting information on the implementation of the cooperative teaching 

program would require taking account of four sets of factors. 

a. The tyoes and amounts of data needed and the time required by the 
investigator and by respondents during the data-gathering process* 

b* The order in which different sets af data should be obtained in case 
some of it would set the stage for gathering other information* 

c* The periods of availability of needed data from various sources* 

d* The period of time during which all of the data have to be gathered* 

Assuming you^have the task of gathering the data, what types and amounts 

would be needed? Consider these suggestions: 

—Diary records on 5-days' parti<iipation in instruction by each member 
of each of the 12 teams (48 teachers and 12 paraprofessionals) 

— An interview with each team leader on team structure and procedures* 

—Diary records on one week's team leadership by each of the 12 leaders. ^ 

"One day's obsmation of instruction in each of the 12 teams* 

"Observation of one team meeting held by each of 6 teams* 

—Observation of student advising by one member of each of 6 teams* 

—Questionnaire returns from all team menrbers on how they perform thel 
^functions of team planning, student advising, instructing individuals 
*and groups, or performing paraprofessional duties* 

The time required for obtaining these sets of data would relate mainly 

to you* However, when you wanted to conduct interv'^ws with team personnel, 

you would have to find up to an hour when each was free of other duties* This 

time probably could be found during the dally two-hour period when the students 

were studying the special subjects. Presumably, team members could find time 

in their schedules to keep diary records and to fill out questionnaires* 
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Assuming you'had just one month to collect your data, you could make 
a plan for using the 21 or 22 days available to you to obtain all the tyj^^., jf 
data required* Here is the sort of time plan that would meet your requirements. 
It assumes, of course, that you would spend time beforehand in planning your 
data gathering and time aftenvard In analyzing your data. 



TIME ASSIGNED TO DATA-SATHERING TASKS 





DATA-GATHERING TASK 


TIME ALLOTTED 


1. 


Interviewing the vice principal in charge of the 
cooperative teaching program 


3 hours 


2, 


Reviewing reports,, schedules, lesson plans, and other 
writttn matter 


2 days 


3. 


Interviewing 3ach team leader on team structure and 
procedures 


2 days 


4. 


Obtaining questionnaire data from team members {time 
required for ex|jla1n1ng procedures, distributing and 
collect'^itg replies) 


3 hours 


5. 


Obtaining diary records from team members {time 
required for explaining procedures, distributing and 
collecting replies) 


3 hours 


6. 


Observing one day's instruction by each of 12 teams 


12 days 


7. 


Observing a team meeting held by each of 6 teams 


1 day 


8. 


Observing student advising by one member of each of 
6 teams 


1 day 



The performance of the different data*gathering tasks necessarily would 
overlap considerably. However, in general, the task<^ should be performed in 
roughly the order in which they are listed* The first four tasks would give 
you many valuable leads for conducting the ta^ks calling for observatioru 

The time estimates given call for approximately 18 of the 21 or 22 days 
available during one month. Twelve of these days are tied up with observing 
instruction* During the other days, your time would be d'ivided amon^ such 
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tasks as interviewing team leaders, administering questtonnaires, observing 
student advising, or observing team meetings. 

5, Planning how to analyze the implementation data . The purpose of 
assessing program implementation is to identify shortcomtngs in the 
instal/lation of program features as a basis for planning w^s to *^emedy 
those shortcomir,gs. When the implementation data have been obtained, they 
need to be analyzed to arrive at a summary of the shortcomings of 
implementation found. 

The procedure to follow in obtaining this summary of shortcomings 
begins with listing the program's features. Each feature should be broken 
down into sub-features in preparation for pinpointing the shortcomings 
found. To illustrate this, consider the features of the cooperative teaching 
program listed on page 36. Here is how two of the features could be broken 
into sub-features in preparation for the data analysis; 

Feature 9^ Flexible scheduling and grouping 
a* Planned by the team each week 

b. Providing for varying the time different students 
spend on a given subject 

c. Allowing teachers to work with individual students, 
small groups, or usual-sized group*^ 

Feature 11. The aide has these functions 

a* Preparing lesson materials 

b. Grading objective exams 

c* Monitoring the use of learning materials and equipment 

Armed with this list of program features and sub-features, your task now 
becomes that of setting up data analysis and summary sheets for each feature. 
In the case of the cooperative teaching illustration, you would need to have 
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separate data analysts sheets for the 12*teains stnce shortcomtng? are apt to 
be true of certain teams but not others. 

Begin your analysis with the structural features of the program under 
study. This part of the analysis usually takes less time and Judgment than 
the analysis of implementation of procedural aspects of the program. It usually 
is found that shortcomings in program implementation are less frequent with the 
structural aspects of a program than with the procedural ones. However, v/hen 
structural features are not implemented, this almost guarantees that the related 
procedural features will not be well implemented, if they are at all. For 
example, if a curriculum program calls for the use of certain supplementary 
learning materials and those materials are not provided, teachers will not be 
able to perform those functions involving the use of the suppl'^Tientary 
materials. 

Even with regard to structural features of a program, you will find 
contradictory evidence when you compare data obtained by different methods or 
from different Informants. Resolving such contradictions calls for deciding 
which evidence is most trustworthy. Your direct observations are lifcely to be 
more accurate fo r the time of your observation than data from any other source. 
However, since you could not observe more than for a very brief time, your 
observations may not be representative of what normally happens. Generally, 
when data from two or more sources are in agreement, you are on safer ground 
trusting the data than when there is a lack of such agreement. 

In obtaining evidence on shortcomings in program functioning, a 
combination of diary information and your own observations should giVe you 
the best basis for drawing conclusions on the implementation of procedures. 
The diary information, if it reports "what happened today'* is likely to be 
of great value in giving the "big picture*' though people tend to report their 
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own perfomance tn tlie most favorable Ught ih^t ts conststent with 
truthfulness. Your observations will offer both a check on diary reports and 
a more detailed picture of what happened* 

Tour plai^ for data analysis should call for a suinmary statement on the 
shortcomings of program implementation identified in your investigation. For 
each shortcoming found, your summary should identify it as accurately as 
possible, should indicate its degree, and should specify its extent. For 
example, using the cooperative teaching illustration, if jfou found that non- 
graded advancement was poorly implemented, your summary should point out the 
curriculum areas where it was not effectively practiced; should designate the 
ways that instruction failed io provide for different rates of advancement; 
and should indicate how widespread the failure was (e.g., true of 9 of the 
12 teams). 

Your plan for data analysis should provide for checking your conclusions 
with key local personnel. (Of course, if you had been working with local 
personnel throughout the assessment ofj^ implementation of the program, this 
inside interpretation of events would have been provided all along.) The 
^advanta^e of this check of your conclusions is that, very often, local 
personnel can explain situations that led you to distort your data and draw 
incorrect conclusions. 
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Briefly, Objecttve 2 was concerned with developtng a plan for 

assessing the implementation an an educational program and it involved you 

in the activities listed below* 

1* Identifying methods for obtaining data on the implementaticn of 
program features: 

a* Interview 

b/ Questionnaire or inventory 

c. Analysis of diary or journal entries 

d. . Analysis of records or reports 

e. "Observation 

2* Setting up a plan for assessing the implementation of program 
features: 

a* Specifying the purposes and scope of the assessment of 
implementation 

Obtaining a representative sample of data on program 
implementation 

c* Choosing methods for getting data on implementation of 
program features 

d* Setting up a data-gathering schedule 

e* Planning how to analyze the Implementation data 

An illustration of a one-month investigation of a cooperative teaching 

program was used as a vehicle for examining the various aspects of planning 

such an assessment* 
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Objective 3. Design a plan for conducting a causal analysis of shortcomings 
tn the implementation of an educational program* 

Once you have identified the shortcomings of implementation of the 
program under study and have developed a Itst of these shortcomings for each 
program feature, you must then consider the factors probably causing those 
problems. Discovering these causes is an essential step toward Improving 
implementation of the program. 
Listing Pc^ential Causes of Shortcomings Identified 

In undertaking a causal analysis of shortcomings of implementation, the 
first task is to draw up a list of potential causes for each shortcoming. 
Using your knowledge of school systems, think of factors which Tiight affect 
the implementation of the program feature in question* Does the system 
provide sufficient materials and equipment for the program^s operation? Do 
the custodians discourage moving furniture around in the classrooms with the 
result that flexibility in grouping is hampered? Do line staff favor and 
support the program? Etc* 

Spend some time reviewing your notes and checking your memory for cues 
you picked up while gathering data on program implementation. Also, consider 
the suggestions given to you by program personnel when they examine your list 
of shortcomings* AHow your imagination to come into play as a valid means 
of brainstorming potential causes of shortcomings in the program* 

Exercise 6 asks you to develop a set of likely causes for a shortcoming 
in implementing ttie role of student advisor by members of a teaching team* In 
the list of features of the Bigtown cooperative teaching program, each team 
teacher served as advisor to about thirty students* The role called for 
seeing each student regularly to advise on progress in both basic and special 
subjects. 
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Suppose that you found the following items to be frequent failures in 
the performance of this role; 

a* Host of the teachers met with students only in groups, or 
on an individual basis only when the student or his/her 
teachers reported sonie special problem. 

b. Host of the teachers dtd not attempt to help individual students 
view their progress and problems at school and work with them 

to arrive at solutions. 

c. Host of the team teachers did not concern themselves with 
their advisees' progress outstde of the four basic curriculum 
areas taught by the team. 

With these failures in mind, turn to the worksheet for Exercise 6. Think 
of undesirable conditions or incorrect procedures that may be present and of 
desirable conditions or procedures that may be absent. When you are finished 
with the exercise, check your work with the Answer Key that follows. 
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EXERCISE 6 - ttORKSHEET 
Ltst of Likely Csuses for a Fatliire to Advtse Indtvtdual Students- 

Dtrecttons : Your task ts to think of plausible reasons why a majority of 
team teachers failed to perform adequately the role of advisor to 30 
students in the team. What may have been causes of this shortcoming? — 

' Scheduling? Grouping? Lack of directions? Conflict v/ith other duties? 
Offer the possible causes that occur to you in the space below. 
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BtERCtSE 6 « ANSWER KVC 

List of Ltkely Causes for a Pailure to rtdvfse Indivfclual Students 

There are many possible causes for the team teachers' failure to fulfill 
the assigned role in advising individual students. Putting together a list of 
likely causes requires knov/ledge of schools, teachers, and team teaching, as 
v/ell as an active Imagination* The list below Is not the answer to Exercise 6. 
Instead^ It represents one set of plausible reasons. 



The failure of many team teachers to advise Individual students regularly on 
progress In both basic and special subjects may be due In part to: 

The lack of a clear and specific plan for the advisory function 

Schedule conflicts where advising a student ;:ould require Interrupting 
other scneduled duties 

Reluctance of teachers of special subjects to be cooperative In reporting 
a student'? progress and problems 

Teachers' lack of belief In the Importance of advising students on their 
total program 

-Teachers' preference for seeing students primarily or only when problems 
arose 

Lack of training of the team teachers In the conduct of student advising 
A preference of teachers for group rather than Individual advisory sessions 

Remember, the above Is In no sense an official or "right" list of likely 
causes for shortcomingsv In the conduct of the advisory function. It is just 
one suggested list. 
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Investigating Potential Causes of Shortcomtngs 

To this point, you have studied the ftrst part of the causal analysis of 
shortcomtngs—prepartng Usts of plausible causes of the shortcomings that have 
been identified* The next task ts that of planning arf Investigation to 
determtne which of the plausible causes listed, or others not yet identified, 
are actual causes . 

Planning an investigation into causes ^f a shortcomtng calls for taking 
each potential cause listed in turn and specifying how data could be obtained 
to check on whether it actually was at least a partial cause of the 
shortcoming* 

In planning your investigation, you will need to decide where to turn for 
data on each potential cause to be checked* If the program you are seeking 
to improve has a coordinator or director, your first step should be to seek 
information from this source on the validity of any of the potential causes 
you have listed C6r othei^s the coordinator considers to be operating)* Also, 
the coordinator can offer valuable advice on where to turn for the evidence 
you need. Your search for tJata on actual causes for shortcomings in program 
implementation is important and justifies your t,aking as much time as you need 
to obtain a good "fix'' on the causal factors that are producing faulty 
implementation* 

Planning a causal search can be illustrated with a list of likely 
causes for a failure to provide for nongraded advancement in the cooperative 
teaching program* These plausible causes are listed on the following pages 
with suggestions for how each one might be investigated* You are invited to 
add your own ideas to those suggested* 
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Patlure to Provide Nongraded Advancement 

Potential^ cause: The cooperative teaching plan may fail to specify how 
nongradtng fs to be accompltsltedK 

Suggestions on search for evidence : 

Ask the vice-prtncipal in charge of the program for a written 
description of the cooperative teaching program and check ft for 
provisions' on planning and conducting nongrading* 

If there is no written description of program features related to 
nongrading, ask for an oral account and ask how team members were 
informed* 

Your .own suggestions: 



Potential cause; There may be a lack of individualized learning materials 
* , needed if students are to advance at individual rates* 

Suggestions on search for evidence : 

Ask the vice-principal and some of the teachers to show you what 
materials are available that students can work with independently; 
also ask how these materials are assigned and used* 

Observe instruction in a few situations where individualized 
materials might be found in use. 

Your own suggestions: 



Potential cause: Teachers may fail .to favor nongrading either because 
they consider it undesirable or very hard to achieve* 

Suggestions on search for evidence : 

Ask the vice-principal and some bf the team leaders to report their 
attitudes about nongrading (desirability and difficulty) and also 
to report how they think teachers in the team feel about nongrading* 

Ask a sample of team teachers what their views and feelings about 
nongrading are* 

Your own suggestions : 
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Potential cause; There ui^ 5e a lack of planning for nongrading when the 
team sets up plans for the next week^s scheduling and 
grouping^ 

Suggestions on search for evidence : 

Ask several team leaders to show you plans for the current week and 
to point out to you w^s in which those plans make provisions for 
students of the same <irade advancing at different rates. 

Your own suggestions: 



Potential cause; Problems posed by multi-disciplinary instruction may make 
tt difficult at the^me time to foster nongrading. 



Suggestions on search foT* evidence : 

Ask team leaders to show you plans made for multi-disciplinary 
instruction and to point out how these plans permit or hinder 
nong^dlng. 

Observe multi-disciplirlary 'instruction as conducted by one teacher 
tn each of several teams and determine whether any nongrading is 
provided for in at least one of the disciplines involved* 

Ask a number of teachers whether they can let students advance at 
different rates in at least one of the disciplines involved when 
they do multi-dtsciplinary teaching. 

- Your own suggestions : 



Potential cause: There may be a lack of needed teacher education on 

nongrading with the result that teachers do not know how 
to conduct it. 

Suggestions on search for evidence : 

Ask the vice-principal what sort of teacher education for nongrading 
was given and what sort of supervision is provided to assist teachers 
in understanding and practicing nongraded instruction. 

Ask a sample of team leaders and te^ hers what they understand 
nongrading t& be, how it can be conductr^d, and v.'Viat problems they see 
in trying to conduct it. 

Your own suggestions ; 
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Having reviewed the example of Investigating causes for failure to 
provi'de for nongraded advancement, try Exercise 7, which asks you to carry 
out a similar analysis for the role of advising students. Potential causes 
for failure are provided; yoa are asked to suggest sources for obtaining 
evidence about them* 

^ Check the Answer Key that follows after completing the exercise* 

t 
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EXERCISE 7 - WORKSHEET " 
Search for Evidence on Likely Causes of Faulty Program In^letnentation 



' Directions : For each of the follwing potential causes of teachers* failure 
to advise the students assigned to them {regularly, individually, and on 
their total programs), v/rite your recommendations for sources of evidence. 



POTENTIAL CAUSE OF 




FAULH ADVISING 


SUGGESTED SOURCES OF EVI0ENCE 



Ttie program plan may lack specific 
provisions for conducting 
individual advisory conferences. 



There may be schedule conflicts so 
that advising a student would 
require interrupting other 
schs^duled duties* 



Teachers of special subjects may be 
reluct€int to cooperate in reporting 
a student*s progress and problems 
to his advisor. 
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POTENTIAL CAUSE OF - 
FAULTY AOVISING 


SUGGESTEO SOURCES OF EVIOENCE 


Teachers may doubt the importance 
of advising students regularly 
on their/ total programs. 


/ 


Teachers may wish to see students 
when they have problems rather ; 
than on a regular basis* 

i 





Teachers may prefer to advise 
students in groups rather ths^n 
in individual conferences* 

\ 

\ 




Teachers may lack the needed 
training in planning and 
advising individual students* 


- 
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^^ EXERCISE 7 - ANSWER KEY 

Search for Evidence on Likely Causes of Faulty Program Implementation 

Probably you have found in performing Exercise 7 that the search for 
evidence on causes of faulty implementation should be very similar for both 
shortcomings in nongrading instruction and in student advising. 

Obtaining evidence on how well the program plan described the conduct 
of student advising calls for asking to see the program plan or asking for an 
oral report on its provisions if they were not set in writing. 

Teachers' problems with scheduling advisory conferences could be 
investigatec^ by asking team leaders or teachers to show you such conflicts in 
their schedules and to point out difficulties they had:in resolving those 
conflicts. 

Teachers' attitudes and preferences with regard to conducting advisory 
conferences dan be learned through interviews, although reports by the vice 
principal or team leaders can be valuable as well. 

Determining whether teachers had received the training they needed for 
planning and conducting] student advising requires evidence from the vice 
principal, team leaders, and teachers themselves. If te=ichers cannot explain 
the purposes and procedures of the advisory function, their training could 
not have been adequate. 
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SUMMARY 

Objective 3 has dealt with analyzing shortcomings in the implementation 
of program features to dete^^mine their causes* Two steps tn the process 
were reviewed: 

1, listing likely causes for each shortcoming 

2, Gathering data on likely causes to determine which are actual cabses 
The purpose of the causal analysis i£ to focus activities directed toward 
improving program implementation. Thus, if the failure to implement certain 
program features can be attributed to a lack of the needed staff training, 
this provides a specific basis for planning improvements in implementation. 

Once the needed causal data were obtained, you would be ready to set 
about develoj>ing a plan for overcoming shortcomings in program Implementation* 
This is the function of Objectives 4 and 5 of this jjnit* 
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Objective 4* Oesign a plan for identifying and evaluating resources 
Hkely to overcome the shortcomings identified*. 

Having identified shortcomings of implementation of features of a 
local educational program, and having probed factors that are probable 
causes for these shortcomings: youi^ Tiext task is to survey various "resources" 
that could be employed to remedy the shortcomings of implementation* 
Remember, the purpose is not to locate another program that might replace 
the present one. Rather, the purpose is to make the present program function 
better, [Of course it could turn out that the present program, even when 
effectively implemented, failed to meet its objectives successfully and 
should be abandoned in favor of some other program* In that event. Unit 8 
of this training program. Selecting an Educational Improvement Program ^ 
would be relevant,) 

By resource is* meant any procedure or materials that could overcome 
a shortcoming in impleEiientation. A resources search means consulting various 
sources that might provide information about useful resources* The resources 
you are seeking may be discovered anywhere, including the local situation* 

How shouTd you plan a resources search to locate and evaluate resources 
that could overcome shortcomings of implementation of a local educational 
program? It is a safe assumption that your time for this will he limited 
and that you will need to make strategic use of the sources available to 
you. Basically, your task is to obtain information about procedures or 
materials that have been used successfully elsewhere in implementing program 
features that are not being effectively implemented in the local program 
you are seeking to strengthen. Where should you turn for such information? 

Your first step should be to seek Informants vMo can give you expert 
advice, either on how to solve the implementation problems you are concerned 
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with, or on where to turn for the information you need* Where can you find 
such experts? Field consultants of your state education department (including 
its intermediate or county units) provide one key source. They may be able 
to direct you to school districts in the state that have dealt with some of 
the same implementation problems you are facing* Phone calls or visits to 
these districts then could follow. Also, in numerous states, there are 
education information centers (often associated with the state education 
department) whose job is to locate and report infonnation on any aspect of 
educational practice* These centers have full access to the literature of 
education as well as extensive knowledge about educational agencies or 
individuals v/ho can offer expert advice or assistance with various problems. 
Another key place to-turn is the school education of a neighboring 
university whose faculty is apt to contain one or more members with expert 
knowledge of the sort of prograri you are concerned with* 

The Educational Resources Infomation Center (ERIC) of the U. Office 
of Education is a source of reports on mny topics in education* Your state 
edaeation department can tell you how to make use of this source, in case 
you are not already familiar with it. ^ In addition, there are 16 
Clearinghouses, each charged with maintaining up-to-date abstracts of 
information within its area of specialization - career education, reading and 
communication skills, teacher education, etc. A list of the ERIC 
Clearinghouses is given on the following page. 

In every area of educational practice, there Is an extensive literature 
describ'^flg different programs, procedures, materials, or equipment that have 
been employed. Journals of educational organizations are especially apt to 
contain the specific information you need on how to solve implementation 
problejiis. For example, The American School Board Journal . January 1975 issue, 
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ERIC Clearinghouses llncli of llic 16 clcaritifjiouscs, as welt as acquiring* rcvicwiitg. 
absmctiii|*« aiid inJcvtn*: the docunicnt^ .uiiuuincctt in Research in L'ducatton, aljio picpnrcs bibliojir-iphics and 
inlcrpiciivc siiiiiiiunci iif icsc:ik1i ap|tcar in Rcscorch iit Ldnc^ilioii hukI nrc {Ji^L^ininjtcd through [lie URIC 

DcK'itinCiil Uopji*dtiv [iun Service, cle.iiinfJioiJses luvc Ijiiiiteil resources for provnirng deloilcd replica lo 

jiUliiiricN ffif inroriiMlihii on specific iupiv>, cdncotors ^tQ iirf^ed to stiliscribc to Rcsc»rcU h) Edueotioii ;itid oliljiii the 
Dlhcr VAWC puhlicaiiuns lo search for desired fnTornialioii. 

1:R1C Cle:iriiij;liouscs jnd Ihcir addresses ore listed below: 



ERIC CLEARINGHOUSES ON 



CARKKti KI)UC\Tt(>S 
2(MrMhcltl:ill 

cOl<^sK|J^f;.\\l>PKRS(>^■^KJ.s^u^vta:s 

Univcrvii>of ^lichirin 
Seliool of } JMC-ilKin lidilJin^. Ht'ori 2(08 
tCn^l L'niwi^it) K Sonll) I ni^orvnySl^. 
Ann A'bcv. Mjcl)i^:tn ^IKV^'I ' 

mS".At>VANrA.CjKI* 
Icocticr^Olletv 
('Columbia IJntvvtvity 

NcwYiirk.NewYi'^rK 10027 

i':.\ttl.V('lltlJ>tK>Ot>KjnCATJON 
Uiiivcivity oi JNiDOi^ 
RU^ tVnii>vlA4on \v^. 

KDl't;ATI()N \l MASAGi:.\JKVT 
iJoi^vr^tty ot ()(vfi\o 

ii.\N(>iCAi'n':t>AM>(;ii'Ti:i>(niJJ>RHN 

l^cCoiincilttv I \vvTt^ont<iri)iM(vn 
1920 A^^t>eMiiiMi Drive 
RcVlOn. Virt;jniu220^t 



JlKniKR KDlT UtON 
(icoffc W;ivliingion Uni\cFvlty 
Oiic tl<ipoul t'ircli\ N.\V„ Suite 61(1 
\V:i\liiO|;Uin. !>,(', ICiW 

fNTOKM \Ti(>vn^:sonuT:s 

Si.inti^fft CVritv-r Iof Kev^.irvh jmf 



Univcrviiy c^f Cilifornu .ii J*o^ Aofcles 
Pi'^weli l.ibrjii) . Room % 
405 lfilf:iFd Ave. 

Ani:icle\XalifoF(ii:i'nKl24 

tANOUAG^:^; aNJ> UNGUtSTtCS 
Ccnier for Apptic'd Ungtiistie^ 
1611 UouU Kent Sued 
Ar1ingion« Vir^inin 22209 

RKAI>I\G ANDC()\1Ml*MC\rK>VSKll,l^S 
Riiional Cminctf of Tei^elici^ of Tn^livl) 
it II Ken)-oii tiwid 
UFtiitnO. Httnois^lflDI 

HURAi. ICtJCCATtOV AM>S\fAtJ.SCttO(>i.,S 
New Mexico Siaie Ufiiver^h) 
Box SAP 

Ui Cnieci, New MeKieof^HOO^ 

SCJV;NTK,\tATtVKMATH'S.ASI>KNVlR(>SMt:MAL 

EtUJCATlON 

Otiio Stale UnivcF^hy 

400 t.iiieotn TovieF 

Cobmt>iK.Ol«o A^2I0 

SOaAl-STtHlHS/SOCIM.SCtKNCBKDlTATIOS 
Soeial Science MOiie:iiion C^^t'.^tutiiim. tnc^ 
S^Jt tirciiidw^ty 
notilJcr. Cotor:idi) 

THACttl-tJ t:i)UCATiON 
Anierie:iii A>'^oci«iit^n ol Caitcpe^ foF 

Tefietier t^duv^nton 
One F>ufHini ("ucIc.NAV . Stnic^tA 
Warliiotm».l>.C 200,U 



mrs.MItASTKIIVHV^u AM) I 
t^diietitionjl levtJOi; Scf\ lee 
pFineeic^iK New Jc fvcy (i8S40 



:\ALVATtOV 
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contains an article descrlbtng ways different districts have attempted to 
deal wtth violence In the schools* Turning to the Education Index is a good 
way of gaining access to articles reporting problem-solving approaches that 
have beei"* i**^ed In different school districts* 

In a good many current areas of concern. Education U.S*A* Special Reports, 
published by the National School Public Relations Association, give up-to-date 
information on school practices* These reports are available, for example. 
In the areas of Informal ecfucation, alternative schools, student rights, 
drugs, vandalism and violence, individualization, vocational education, and 
differentiated school staffing. 

Since the problems you are concerned with Involve- Improving the 
Implementation of a school program, rather than with Introducing a new 
program, you will need to cull the information contained in tha literature 
carefully to glean Items that focus on program Implementation, It Is 
unfortunate that there are relatively few books or articles that deal 
primarily with overcoming problems of implementation. However, if you have 
clearly l^lentlfled shortcomings of program Implementation, and If you have 
pinpointed likely causes for those shortcomings, a careful search of the 
literature will probably yield numerous approaches that school districts 
have used to make;^prograins work more effectively* 

An tllustratlon will help make youi* task clear* Consider thfi problem 
of correcting shortcomings In Implementing nongradtng In the ^cooperative 
teaching program at Bigtown Intermediate School as dealt with In Objectivas 
2 and 3, for convenience, let us limit the illustration to nongrading 
mathematics instruction where the purpose Is to have each student work at a 
level appropriate to his present advancement, and progress at whatever pace 
he can master learning tasks. Assume that your analysis of Implementation 
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has Indicated the following: 

Sho rtcomings of implementatipn 

1, Students are grouped by general achievement level and all students 
in a class are taught the same learning unit at a given time, 

2, In a class, all students work on a unit using the same learning 
materials. 

3, Students in a class all are ntoved to thennext learning unit at the 
same time even though some have not mastered the unit just studied. 

Causes of faulty implementation 

1* Teachers lack sub-tests for measuring students' level of mastery 
of each unit in the math sequence^ 

2. Thece is only one s6t of materials for studying a learning unit, 
rather than alternative sets of materials (including manipulative 
materials) for a given unit* 

3. The math teachers have not had in-service training on methods of 
adapting instruction to individual differences among students 
within a class. 

From this analysis, you need to seek resources that will provide achievement 
sub-tests, alternative learning materials, and appropriate in-service 
training for teachers. Thus, your efforts should be to find "^ays of 
overcoming causes of faulty implementation as ways of overcoming the 
shortcomings noted. 

If you are not alreac^y familiar with the literature on nongr^ding, 
a good way to start is to review one or more of the basic treatments of the 
subject. The following four books .^re important general references: 

r 

Goodlad, J,I,, & Anderson, R.H, The Nonqraded Elementary School . (Rev. ed' ) 
New York: Harcourt, Brace, & World, 1963. 

Brown, B, F, The Nongraded High School . Englev/ood Cliffs, N.J.: Prentice- 
Hall, 1963, 

^ The Appropriate Placement School. West Nyack, N.Y.: Parkefc, 

1965. 

Hillson, M. , & Bongo, J, Con tinuous-Progress Education. Palo Alto, 
California: Science Research Associates, 1971. 
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In reviev/ing any of these volumes, you will find relatively little that 
is specific about problems of implementation. Yet careful skimming of their 
contents will give you key ideas on how features of nongrading can be put 
into operation. Thus, Brown's book on nongrading in the high school presents 
a ''multi-phase" approach that adapts to the learning capabilities and styles 
of different students, and the Hil Ison-Bongo book contains outlines for 
in-service workshops for teachers (pages 135-152). 

Three sources of information probably will prove useful: consulting 
with experts to whom you are referred (perhaps by consultants from your state 
education department), visits to school districts employing nongraded programs 
at the intermediate level, and descrtptions of nongraded practices contained 
in education journals. For guidelines on achievement sub-tests geared to 
.curriculum units, some of the best sources are the descriptions of 
individualized programs such as Individually Prescribed Instruction (IPI) where 
each learning unit (fn mathematics, reading, spelling, or science) has its 
pretest and posttest for nieasuring precisely the achievement of learning 
objectives. (See Teaching in IPI Mathematics: A Multi-Madia Program of Teacher 
Preparation. Volume 1. Diagnosing and Prescribing for Individualized 
Instruction . New York: New Century, 1972.) 

Let us suppose that your search turned up a half-dozen or more resources 
for correcting each of the three causes judged to be limiting the implementation 
of nongrading. Suppose that your reading, interviews, and visits located a 
number of nongraded math programs eqt* oped v/ith achievement sub-tests geared 
to learning units, several intermediate-level math programs providing 

alternative materials for study of a given unit, and several in-service 
programs designed to train teachers in nongrftding math instruction. What 
procedures should you follow in evaluating these resources for use in improvin*^ 
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the impleTnentation of nongrading in your local instructional program? 

Three criteria are important in evaluating any resources for inclusion 
in your approach to improving:,the tmplenientation of the local program* These 
are relevance to local needs, evidence of effectiveness , and feasibility of 
local adoption* By way of example, an in-service teacher education program 
on nongrading might be judged relevant through covering all aspects of 
nongraded instruction where the local teachers were found to be deficient. 
It might be judged effective on the basis of evidence of students' individual 
progress in math, of teacher satisfaction, and of administrators* approval. 
However, it might be judged not feasible of local adoption because it required 
more time and money to conduct than the local dtstrfcT could provide* 

In evaluating a resource for adoption, it often will be the case that 
the resource contains elements that would meet local needs if adapted to the 
specific features of the local program. For example, the achievement sub-tests 
of math Program X might proVide an excellent model for use in the local 
situation; however, the local math sequence might be different from that in 
Program X so the model would need to be adapted by building sub-tests geared . 
to the local curriculum. 

Conducting this process of evaluating potential resoances for adoption 
ts a task that tests anyone's capabilities to the limit* Evon an expert in 
evaluation is almost certain to /ind that there ts not enough good evidence 
available to make confident judgments. Your task uill be to make the best 
use of evidence in making your judgments, realizing that some 6f the evidence 
may be untrustwoHhy or inadequate* Judging rel evance is a matter of matching 
elements of the resource with the needs you need to meet* TIius, suppose that 
teachers need to learn how to locate children in a nongraded curriculum 
sequence, hov^ to provide them with the learntng tasks they are ready to study, 

IS) 
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how to assi*;t them as they study on an individual or group basis, and how to 
dettjrmine when students are ready to proceed to the next learning task. Do 
you find in accounts of the teacher-educ<ition resource that provisions for 
thes*: elements are included? 

Judging the g^ectiveness of the resource may be extremely difficult 
since it very often is true that there are no data on this criterion except 
for opinions of those who have used the resource^ You may have to rely on 
these opinions, in the absence of objective evidence* In this case, it is 
important that you note the sort of evidence available to you and state your 
degree of confidence in your judgment. 

^ Judging the feasibility of adopting the resource in the local situation 
calls for checking for evidence that the time, money^ and leadership needed are 
available, and that attitudes about adopting the r^^source are favorable. 
Generally, after you have made your initial estimates of feasibility, it is 
Important to obtain the reactions of key decision makers who will be involved 
in adopting the resource, and of key participants such as principals and 
teachers^ 

The end product of your resources search should be a priority listing 
of resour<^es related to each of the causes of faulty implementation you were 
seeking to correct* Under each resource listed, you should have information 
on that resource's relevance, effectiveness, and feasibility. With these 
listings in hand, you would be ready to undertake the final task in the 
process of planning to improve the implementation of the local educational 
program. This task - actually planning to accomplish the needed improvements - 
is the substance of Objective 5 of this unit. 

To give you practice in thinking through the process of conducting a 
resources search, Exercise 8 foUavs. You are asked to create your own 
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example by select^no a (real of simulated) local educational program, 
ind^cat^ng one shortcoming of its implementation, and specifying one cause 
for that shortcoming* The program you choose can be of anv tvoe and any 
level of schooling: the conduct of Instruction using a new curriculum in a 
subject such as reading, math, or science; a program of school /community 
relations; a program to deal with discipline problems; a modular-scheduling^ 
program; a program employing teacher aides; etc* The exercise calls on you 
to outline a process you might follow in turning to different sources in 
search of resources that could remedy the shortcoming by dealing with its 
cause* Finally; the exercise calls on you to describe a (real or simulated) 
resource that could remove the shortcoming, then to indicate how you would 
evaluate it for local adoption* 

When you complete the exercise, turn to the A'rtswer Key that follows the 
Work?>heet to check your work with the suggestions contained in the Answer Key. 
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EXERCISE 8 " WORKSHEET 
^* Outlining a Plan for Conducting a Resources Search 

Directions : First, select a lineal w^;^ (fl) and indicate one shortcoming 
of implementa^tion (B)^*4jidxne cau^ fbr-^hat shortcoming (c). (These can 
be either real or simulaJleo.) NexV-Atlina the process you might follow 
in turning to different sources to loc^t&^esources that could remedy the 
shortcOTiing (D). Finally, take one {real or simulated) resouice (E) and 
indicate How you i^uld evaluate it for local adoption (F). When you 
COTiplete the exercijse, turn to the answer key for guidelines to use in 
checking your work under 0 and F. 

A. BRIEF DESCRIPTION OF LOCAL PROGRAM CHOSEN ; 



B. ONE SHORTCOMING OF PROGRAM IMPLEMENTATION 

* 

V 

C. ONE CAUSE OF iHE SHORTCOMING 



D. PROCEDURE FOR RESOURCES SEARCH 



(Continue on next page if more space is ne^^def^} 
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Exercise 8 - Worksheet - Z 
D (Cont'd.) 



* I. ONE RESOURCE TO REMEDY THE SHORTCOMING 

/ 



F. PROCEDURE FOR EVALUATING THE RESOURCE FOR LOCAL ADOPTION 
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EXERCISE B " ANSWER KEY 



Outltning a Plan for Conducting a Resources Search 



Explanation : Since you chose your own example, this ans\ver key simple 
offers you some checkpoints for examining your ansv/ers under Procedure 
for Resources Search and Procedure for Eval^jating the Resource . If 
you left out key steps in either procedure, it probably will be useful for 
yofj to review the contents under Objective !4< 

d7 PROCEDURE FOR RESOURCES SEARCH ; ' 

Your answer should deal v/ith the follov/ing: 

How you v/ill seek expert advice in startiiiQ your search. How will you 
seek such advice from informants in the state education department, in 
\ education information centers, in local universities, etc.? 

How will you conduct a literature search for resources, using indexes, 
following up lists of references in articles or books, etc. 

How you will arrange telephone interviews or visits with people who 
have made use of resources in the area you are concerned with. 



How you will judge the rele vance of the resource to local needs by 
matching specific needs with features of the resource. 

How you ;/ill judge the effectiveness of the resource using any 
objective data available and using opinions offered by those who have 
made use of the resource. 

How you will judge the feasibility of adopting the resource in the lotal 
sit'jation, taking account cV requirements for time, tncney, and leadership 
as well as the attitudes of key persons v/ho will be involved. 



F. PROCEDURE FOR EVALUATING THE RESOURCE FOR LOCAL ADOPTION 
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SUMMARY 

Objective 4 has dealt with the task of locating and evaluating 
resources that could remedy shortcomings of Implementing a local educational 
program. Mlmost without exception, a variety of approaches have been 
developed and tried in various locations for meeting any local need* The 
problem 1s to locate infornation about these approaches and to judge its 
applicab11ity*l ocally* 

The content under this objective has covered maklr.g use of different 

sources of information: 

Experts either to provide specific information about resources, or 
guidance in turning to appropriate sources. 

Files of the Education Resources Information Center (ERiC) or ERIC 
Clearinghouses. 

Education Information centers, or s1^.:ilar resource centers. 

Local agencies that have employed any resources related to local needs. 

In evaluating resources for local adoption, this objective ha'i considered 

the use of three criteria: 

Relevance of a resource to local needs for 1 .firoving implementation. 
Effectiveness of tne resource In meeting needs similar to local needs, 
Feasibility of adopting a given resource In the local situation* 
Exercise 8 has offer d practice In planning a resources search and 

evaluation, using an example of your own choosing. 
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Objective 5. Design a plan for improving the implementation-^of an 
educational progranh 

To this point, you have studied four tasfcR that are basic to improving 
the implementation of any local educational program. You have examined ho.v 
to identify and lirt the key features of a program (Objective 1)* Next, you 
studied how to design a plan for assessing the extent to which the features 
of a program haive been implemented (Objective Z). The thir:J tt^k involved 
study of how to design a plan for a causal analysis of shortcomings in 
program implementation (Objective 3). The fourth task dealt with identifying 
and evaluating resources that could remedy shortcomings of implementation 
(Objective 4)* Now, in Objective 5, you will study how to design a plan for 
correcting thtse shortcomings by fc^nploying resources that attack their causes. 

Note that the focus of Objective 5 is not on designing a change program* 
Instead, it focuses on efforts to inprove the implementdtion of whatever 
program already is in operation, whether it t$ traditional or innovative. 

Planning an improvement approach breaks down into three steps: selecting 
the resources to be included, designing at pattern for using these resources, 
and designing a way of implementing the pattern^ The entry point is the 
information you obtained under Objective 4 oi the relevance, effectiveness, 
and feasibility of adopting the resources examined* It v/ill be helpful to 
study the three Steps in the planning process v/ith use of an illustration* The 
illustration we will use involves a program of independent study in algebra 
and geometry at Freeville High SchooK (The data presented are simulated*) 

Freevllle High .School is in an integrated s»»burban district on the fringe 
of a large city* Its stuoent body is largely from middle-class homes and is 
ZZ per cent black* The school serves 2500 students, A program introduced 
tKO yt^ars ago allows all students to study introductory algebra and geometry 
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either in the traditional way or through independent study* Jhose who request 
independent study are interviewed by an instructor and, if they appear to have 
a serious interest tn studying on tUeir own, are admitted to this alternative 
approach. About one*^half of Freevtlle's students representing all ability 
levels, elect to study algebra and geometry through independent study. 

fln evaluation of the independent study approach has revealed serious 
shortcomings* About one-third of those enrolled in independent study request 
a transfer to the traditional method within the first month of the semester. 
Many students who remain in the program iio not progress fast enoggh to complete 
the course taken during the semester. Many request help from the instructor 
several times each week insteatj of ftndtng the scheduled bi-weekly progress 
check of sufftcient help* 

A causal analysis of the prograni*^s shortcoming? has indicated that many 
students admitted to independent study have little interest in the approach; 
it initially appealed to them as *^the easy way out" rather than a challenge* 
Thus, the admission procedures are at fault. Also, many students in the 
program have very limited skills in managing their ovm learning—budgeting 
tiuie, planning lessons^ carrytng out study plans, checking progress, etc. 
Another reason for students^ difficulties is the lack of study materials giving 
specific guidance in what to study, how to study, and how to check one's 
work* Finally, it has been found that three of the six teachers conducting 
the independent study program prefer traditional teachtng and are not 
interested in helping students study on their own. 

tn^'estigatton of resources that could remedy the situation {including 

a literature search, interviews with experts on independent study, and visits 

to programs!, revealed the following resources that seemed of potential value: 

An admissions procedure at Blake High School that assesses students* 
capabilities in independent study and, if weaknesses arc found, teaches 
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key skills In lesson planning* time budgeting, and evaluating one's 
progress* Admission to the progrOT ts granted only to those studeats 
wfth demonstrated skills tn Independent study and a serfous coroltment 
to £hts v/ay of learning* 

An in-service teacher education course tn the Pickwick Public Schools 
that teaches methods of planning and conducting Independent study in 
the^ second^^ry school * 

A set of programmed lessons In Introductory algebra designed for use 
In supplementing Instruction* 

A detailed study guide for Introductory algebra and one for introductory 
geometry acconipanying new text books on these subjects. 

Which of these resources-s^toul d be chosen by Freevllle? Exercise 9 
asks you to make this judgment, based on the (simulated) evidence given below. 

The first test, relevance, you can make without further evidence. Do 
the resources appear to match the causes of program shortcomings given above? 

Tlie second test, effectiveness , calls for your using the following data: 

The admissions procedure at Blake^ according to informants, is a definite 
succe<:s* Teachers swear by it, and the proportion of dropouts from 
Independent study, or of failures to complete a course within a semester. 
Is below 10 per cent* 

The tn-servlce teacher education program at Pickwick was Judged by both 
the instructor and teachers to be helpful in the conduct of the 
Independent study program. The program :Juperv1sor indicated that it 
required $1500 of school funds *^o conduct. 

The programmed materials, according to a department chairman in a school 
district that tried them, had limited usefulness because they did not 
match the "Joc^l curriculum and because some teachers did not believe that 
students could learn much from suc;i materials. 

The use of curricula In algebra and geometry that are equipped with 
detailed study guides was strongly endorsed by several school districts 
that employed them in programs emphasizing Independent study. (Thfise 
data came from telephone Interviews with officials of those districlv*) 

The third test, feasibili ty, calls for you^^ using the following dt^ta based 
on consultation with the Chairman of the Mathematics Department at Freuville 
High School, teachers of algebra and geometry at that school, and ^ne 
Associate Superintendent for Instruction of the school district. 

Freevllle's Associate Superintendent for Instruction, with the support 
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of the Supertntendent, ts strongly conmittted to the program of 
independent study and has up to $5,000 available to strengthen its 
implementation* He ^ees the Blake admissions procedure as a good one 
but v/ould modify Tt by teaching stiUs in tndependent study to 
students v/hen they enter the special program* 

The Chairman of the: Mathematics Department considers in-service education 
of teachers, critical for succe?;s of the program. The Associate 
Superintendent indicates that funds are available for an in-service 
Course on teaching students hot/ to conduct independent study, and 
on guiding students who are studying independently* 

The algebra and geometry teachers are opposed to using programmed 
instructional materials but are eager to make use of the new text books 
that are equipped with guides to independent study. The AssociPite 
Superintendent indicates that funds are available for purchase ^f these 
texts* 

On the worksheet for Exercise 9, indicate v;htch of the four resources 
you would recommend that Fraeville adopt for its program to improve the 
impelementation of the independent study program. Opposite each resource 
listed on the Vorksheet, check v/h(5ther or not you would reco'^mend adoption, 
then check and briefly :> istify your Judgments regarding relevance, 
effectiveness, and feasibility* 
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. EXERCISE 9 - WORKSHEET 

Resources Recommended for Improvtnq preevtlle's Independent Studj^Program 

Directtons : Under each resource Itsted, check whether or not you recommend its 
adoption, then check and JustTfy your Judgments regarding its relevance, 
effectiveness, and feasibtltty of adoption at Freeville. Be sure to write 
in brtef reasor^ for^your Judgments under relevance, etc* 



RESOURCE 


' RECOMMEND 
ADOPTION? 


BASES FOR RECOMMENDATION 


Blake High School's 
admission procedure 


Yes 

No 


Relevant? Yes No 
Effective? Yes No 
Feasible? Yes No 


Pickwick's in-service 
teacher training 


Yes 
No 


Relevant? Yes NO 
Effective? Yes No 
Feasible? Yes N& 


Programmed materials 
in algebra 


Yes 

No 


Relevant? Yes No 
Effective? Yes No 
Feasible? Yes No 


New texts with study 
guides 


Yes 
No 


Relevant? Yes No 
Effective? Yes No 
Feasible? Yes No 

00 



EXERCISE 9 - ANSWER KEY ' ' 

Reso urces Re commendQd fo^ Improving Freeville's IndopcnJcnt StiKly Progr am 

Explanatio n: The suggested ansv^rs given below are based on the data given 
in relation to the fcur resources. These may be different from your answers, 
and you may have sound reasons for judfjmonts different from these. The 
important thing is to think through this choice process carefully, realizing 
that there may be good reasons for your different judgments. 



RESOURCE 


RECOMMEND 
ADOPTION? 


BASES FOR RECOMMENDATION 


Blake High School's 
admission procedure 


Yes ^ 
No 


Relevant? Yes ^ No 
This procedure defini te1y meets some of 
the shortcoming*; in Freeville's program. 

Effective? Yes No 
The evidence from Blake is that this 
procedure has been highly effective. 

Fea';ible? Yes No 
The Freeville Associate Superintendent 
offers financial support for this. 


Pickwick's in-service 
teacher training 


Yes ^ 
No 


Relevant? Yes ^'No 

Frf-eville teachers of independent study 

clearly need training. 
Effective? Yes No 

The Freeville Associate Superintendent 

offers financial support for this. 
Feasible? Yes v No 

Funds are available for the in-service 

courst^. 


Programmed materials 
in algebra 


Yes 

No v/ 


Relevant? Yes V No 
These materials are designed for 
independent ^tud^'. 

Effective? Yes ' No i/ 
The evidence is that these materials were 
not effective where tried^ 

Fedbible? Yes No 
Attitudes of Freeville teachers iridicate 
that these materials should not be adopted 


New texts wi th study 
guides 


Yes / 
No 


Relevant? Yes ^ Mo 
The study guides make these texts well 
suited for use with independent study. 

Effective? Yes ^ No 
School districts using these texts have 
judqed them hiqiily effective. 

Feasible? Yes^ v/ No 

Freeville has funds for purchasing these 
texts and the toacliers favor using them, 
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Once the resources to be used in improving the implementation of the 
current program have been chosen the next planning step is that of designing 
a pattern for making use of them. Continuing the Freeville example, the 
task its that of working out a specific pattern for using the resources chosen 
to strerrgthen the independent study program in algebra and geometry. Suppose 
the decision was made (a) to adopt the Blake procedure for adr^ission to the 
independent study program but to modify it by^providing needed training in 
independent study both before and after admission; (b) to adopt the new texts 
containing study guides; and {c) to provide inr^service^tralning for math 
teachers following the Pickwick model* The design you need to create for 
the changes to be introduced should be a blueprint of what those changes 
would look like if placed in full and effective operatioi*. (Note that this 
is different from creating a plan for implementing thef changes. ) 

The blueprint for the improvement program would need to specify just 
how the Blake admission procedure would work: what procedures {including 
tests and interviews) would be used to determine the student's capabilities 
and intef"c:;t in independent study; what pre-admission training in independent 
study woiMd be given; and what further training in independent study would be 
9iven after admission to the program. The blueprint also would need to 
outline the contents and procedures of the in-service training the math 
teachers would receive in conducting independent study. Finally, the 
blueprint v/ould need to specify how the new texts with study guides would 
be used in the independent study program. 

In setting up specific plans for the changes to bti made, it would be 
important to involve the teachers v/ho would conduct the changes. This would 
make their understanding and acceptance of the changes more likely, and 
would also be the beginning of their training for teaching within the 
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independent study program. 

The final step in planning for the improvement progratrt is that of 
designing a strategy for introducing the changes In the blueprint. This 
requires determining when the program will be placed in operation, listing 
the tasks that must be performed to accomplish implementation of the changes 
to be tflade, identifying resources (personnel, money, time, etc } tc be used 
in preparing to implement the program, and working out a time schedule for 
performing the required tasks. To give^^ou practice in setting up an 
implementation plan. Exercise ]0 follows. 

Exercise 10 calls upon you to set up a time schedule of an implementation 

plan for introducing the changes in the Fr^eville indepe;ident study program. 

It assumes that the program is to be in operation when school opens in 

September. It assumes that,, as of March, the blueprint of the changes to be 

introduced has been completed. The problem is'to outline how the period 

between March and September should be used in getting ready to conduct the 

revised independent study program, Xhe following facts are given you: 

The Associate Superintendent for Instruction is available to supervise 
and supps^rt the process of preparing to Implement the program. 

The Chainrl&n of the Math Of>partment can give the time needed for staff 
in-service training, for developing materials and procedures required 
by the program chancres, and,for*consulting with the math teachers as 
needed . 

The six math teachers will be available for after-school work sessions 
during the spring months and for a two-week workshop during iluly, if 
paid for their time. 

Money is available for the new text series and for paying in-service 
costs . 

Students apply for independent study during March and are evaluated 
for admission before the end of the month. School is over June 2')th. 

Your task in Exercise 10 is to fill in the Worksheet with the activities^ 

required to get ready for ifrtplementing the program in September. What should 
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be done 3ach month, and who would be involved? After completing the Worksheet, 
check your answers with the suggestions offered on the Answer Key* 
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EXERCISE 10 - WORKSHEET 

Time line for Preparinc] to Implement tho Improvement i^rogram 

Directions : Under each month listed, indicate the tasks you would recommend 
for preparing to i\nplement the resources selected to improve the 
program of independent stucy in mathematics at Freeville High School* Also, 
for each task, indicate at the right the staff members you think should be 
dhiefly responsible. (The same task could cover more Zk<\n one month.) 

TASK KEY PERSONS 

March 




i 



Oune 



JulX 



(August-Vacation) ' 
September | 



Do 



/ 
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EXERCISE 10 - ANSWER KEY 

r 

Time Line for (Veparing to Implement the Improvement Program 

E:^p1anation : Tbe syggested time line belovj offers one vjay of prepari .g to 
implement the changes selected to Improve Freeville*s program of independent 
study. Use it merely tocheck your outline, and to remind you of an/ tasks 
you did not include in your pldn. 



March 



April 



; ^ TASK 

Assist teachers in assessing students' readiness 
to undertake independent study in math. 

Purchase the new texts with study guides. 



Segin training of teachers in conducting indepen- 
v^^entjstudy, with empnasis on teaching students 
ills. 



v^Bentistudy, w1" 
the/needed ski 



KEY PERSONS 

Hath Chairaan, 
Assoc. Sup*t* 

Assoc. Sup^t*, 
Math Chai nnan 



Math Chai nnan 



May Train students 1^ skills of independent" study. Math Teachers, 

using methods in Blake approach, as modified fon Math Chairman 
local use. 



June Continue training of students. Decide which Math Teachers, 

students are now ready for the progrSii of Math Chai man 

independent study as shown by their progress* 
in training. 



Jul: 



Hold two-week workshop to train teachers in 
skills required in conducting the program of 
independent study. 



'*^th Chairman , 
Math Teachers 



September Continue training and supervision of teachers 
in conducting independent study. 

^Continue student trainijic; in conduct of 
lindepc ^dent study. 



Do 



Math Chairman 
Math Teachers 
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SUMMARY 

Objective 5 has covered the requirements to be met in-designlng and 
implementing a plan to improve the conduct of a local educational program* 
The planning process has been presented in three steps: 

1, Selecting the components to be included in the Improvement program 

2* Blueprinting the improvement program , 

3* Planning procedures for implementing th&*improvement program 

This training unit ends at this point* The Ideal foUowrup would be 
for you to test your grasp of what the unit has covered by actually providing 
leadership in strengthening the implementation of a local educational program 
in the situation where you work or in a school district where you can offer 
consultant help* 
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POST-ASSESSMENT EXERCISE - UNIT 9 

Planning to Remedy Shortcomings in a Program's Implementation 

Directions : This exercise is the same as the Pre-Assessment Exercise which 
you completed at the beginning of your study of .this unit. Review your 
estimates of mastery of the items on the Pre-Assessment Exercise (page 8); 
you need do only i^iose items on the Post-Assessment Exercise where you 
judged your Pre-Asses silent answers were not adequate. Check your answers 
with the Prp- and Post-Assessment Exercise - Answer Key. 



I.^From the following description, write down a list of the features of the 
MIGS program to be implemented in the Franklin Intermediate School. 

Methods of Inquiry in General Science (MIGS) is a new commercial science ^ 
course for intermediate schools. The two-semester course contains 100 v 
separate project units, each consisting of a study guide, a set of reading 
booklets, and a list of materials, equipment, etc. to be provided by the 
school. There are 20 projects each in mechanics, electricity, chemistry, 
plants, and animals. Each student receives a copy of the Inquiry Methods 
Handbook as a guide in planning projects and preparing project reports. 
The science lab is to be divided into 5 learning stations, one for each 
of the project areas,- and a library center housing reading materials. Each 
learning station is to be equipped with what is required to perform the 
projects in its area and is to provide work space for 10 students, while 
the library center is to provide study space for 15 students. (These figures 
assume a class of 40 students or less.) 

During each semester, each student is required to do one project in each 
area, two or three independently, and the others with a fellow student. 
Each student can do as many additional projects as time permits. The student* 
elects each project with' the instructor's approval. Not more than one stu- 
dent or student pair can work on a given [project at a given time. The 
instructor assists students individually or in pairs. Students present oral 
or written reports on all of their projects as the instructor's main basis 
for evaluating their work. Both instructor and student keep an up-to-date 
record of projects performed and their level of accomplishment. 

List of program fe^itures (Be brief. You may or may not need all the numbers.) 

1. 

2. 

1 



8._ 

10. 
11. 
12. 
13. 
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2. How would you obtain data on whether, the Franklin LS. chose to modify 
its version of MI6S in any way from what has been described? 



3^ Assume 3 science teachers each has four classes in MIGS* What plan would 
you use to assess how well the following procedures were employed in the 
12 classes? (Assume that you gather your data just after a semester, has ended*) 

a* Students elect projects in conference tf/ith the instructor* 




b* Each student does projects in each area each semester, two or three 
individually and the others with another students 



4, Ltst aspects of the classroom setup for HIGS where it is likely that 
shortcomings of implementation will be found* 



• 
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5*^Assume that Teacher A made extensive use of whole-class lectures and 
discussions rather than teaching MIGS cn an individual basis* Whcit are 
a number of Ukely causes of this faulty implementation and hcM would you 
get data on them? 



6* Assume you found that one major cause of Teacher A's failure to follow 
the required procedure was a latk of understanding of how fo Individual- 
fze instruction* What are typ^s of sources you would turn to in seeking 
information on ways of overcoming this failure? 



7* Outline a plan for remedying the shortcoming in Teacher A's instruction 
and state reasons for your choice* 



(Continue on back If you need more space.) 
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PRE- AN[X4^0$T ASSESSMENT EXERCISE - ANSWER KEY 

Explanation : The answers given below, except for the answer to Question 1, 
are meant to offer you a basis for judging the adequacy of the answers 
you have given rather than presenting the pnly correct answers. You 
could correctly have offered different answers* You should judge your 
answers in terms of whether they show an understanding of the planning 
process that compares with that .i::epfesented by the*§nswers given here- 



Q. K List of program features 
1* Materials for two-semester course available - 100 projects, etc, 
^ 5 learning stations provided, each with work sp^ce for 10 students 
3* A library center with space for 15 students 

4* Student does one project in each of 5 areas, 2 or 3 Independently, 
2 or 3 with a fellow student 

5- Students do as many additional projects as their time permits 

6* Only one student or pair of students works on a given project at a time 

7- The instructor offers assistance to individual students or pairs 

8* The instructor evaluates students* work on basis of oral or written 
Reports 

9- The instructor keeps a record of each student's performance and level 
of advancement 

10* The student keeps a record of his/her performance and level of work 

Q* 2, Obtaining data on whether Franklin LS- modified the HIGSi^rogram 

The answer to this question could be obtained by interviewing the person 
responsible for introducing the MIGS program - perhaps the associate 
superintendent for instruction, perhaps the department chairman in science, 

Q> 3a. Evidence that students elect projects In conference with the instructor 

Evidence on this feature could be obtained by interviewing teachers or 

student^, by observing the planning process, or by examining records kept 

of teacher/student planning* Observation would be the most dependable method, 

3b, Evidence that each student does 5 prol^cts a semeiter, one in each 
area, 2 or 3 Individually and the others with a fellow student 

Here the dependable evidence would come from examining records of projects . 
perfonned as kept by teachers and students- 
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Q> 4. list aspects of the classroom setup for HI6S where shortcomings of 
implementation are apt to occur > ^ ^ 

Two aspects of the classroom arrangements need to be checked; whether 
, there are five learning stations, one for each of the five project types, 
and each capable of handling 10 students; and whether the library center 
houses the required materials and can accommodate 15 students. 

Q> 5. Reasons why Teacher fl fails to teach HI6S on an individual basis 

Here are some likely reasons for Teacher A's use of whole^class methods: 

" A preference for teaching groups rather than individual students 

- The lack of training for conducting the individualized approach 

- Students' lack of skills in using the piroject approach 

6> Types of sources of information on improving one's understanding 
of individualized instruction . 

Consult leaders in school systems that emplay the HIGS program- ^ 

Consult an education information center, if one is available. 

Turn to the ERIC file Tor references describing individualization. 

Use the Education Index or other guide to the literature on individualizati 

Consult university faculty members on teacher education for individualizati 

Consult staff members of the state education department. 

7. Plan for remedying Teacher fl's lack of understanding of individual- 
izing instruction . 

The plan you offer in answer to this question is your own. For this 
reason, this answer key can only indicate types of provisions your plan 
should include. 

Your plan should outline the in-service training Teacher A is to receive: 
what should the training include, how should the training be given, who 
wouTd conduct it, and when would it occur? 

Some what components of training would be work on planning students' 
projects, tutoring individual students or pairs, teaching students to 
conduct their projects on their own, and budgeting instructional time. 

Some reasonable how aspects of the training would be selected readings 
about the conduct of individualized project work, observing other ^ 
teachers conduct individualized instruction, and coaching by someone 
who know the MIGS program well. 

The who probably would need to be the chairman of the science department 
or a system-level science coordinator. ^ 

The when would mainly be during the school day but also could include 
after-school study and consultation. 
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Training for Leadership fn Local " 
Educational Improvement programs 

UNIT EVALUATION FORM 

Unit 9, flaalyzfng and Improving the Implementation of a Local Educational 
Program 

* * 

EvaJuatlon by ^ ^ Date_j I ^ 

Position £ Organization ; 



Please give your reactions to this unit by p|»eck1iig and writing In^your 
opinions and reconimendations* Returning thfs fom to Research for Better 
Schools, 1700 Market St,, Philadelphia, Pa. 19103 (Attention; Glen Heathers) 
will help us judge the valjue of the unit as jwell as aiding in its revision, 

A> Your judgment on the importance of a unit on this topic as training for 
leadership in local educational improvement programs* 

Check: Very High High Moderate^ Low Very Low ' 

Your comments; 



Your judgment of the quality of the introductory section of the unit* 

Check; Very High High Moderate Low Very Low 

. ' Your comments: 



C* Your judgment of the adequacy of the set of unit objectives ,^ i 
Check; Very High High Moderate Low Very Low 

What objectives do you recommend omitting? tJhy? 



What objectives do you r^ormiehd adding? Why? 

103 



0* Your judgment on the quality of the Unit contents . 

Check; Very High High Hncierate bow_ 

Your comments; . * - 



V ■ 



Your judgment on the quality of the unit t^xercises . 

Check: Very High High Moderate Low Very Low_ 

Your comments: 



iiatt 9 * Too 
Very Low 



Your judgment on the quality of the UnU pre- and post-assessments . 

Check: Very Highc^ High Moderate ■ Low Very Low 

Your conwients: ^ - ' * - 

■ ** * 

- 1 



G. About how many hours did you take to complete this unit?^^ ^ 

H. How valuable do you judge this unit to be for training each of the follfiwing 
categories of educational leaders? Please enter the appropriate s>.;iboU 

H « Highly valuable. H * M^ de irately valuable. L * Low value 

School system central administrators 

Building principals 

Curriculum coordinators 

F^eld consultants of state education departments 

Graduate students in administration or supervision 

Other: 
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